2024-04-21T10:13:30.666 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-21T10:13:30.666 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} --name leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi --owner scheduled_leonidus@teuthology --verbose --archive /home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584 -- /home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584/orig.config.yaml 2024-04-21T10:13:30.733 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-21T10:13:30.855 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7666584' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: b8ddeda347de6c45510a6df67440a1d298c37030 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1331 sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_b8ddeda347de6c45510a6df67440a1d298c37030/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: b8ddeda347de6c45510a6df67440a1d298c37030 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ2lqbpudOBSVXMl416tiMvWri1zUhLjXS6Y/XKmRBkP4X8ME/S+lHPQhzNdM+J4GV9EfZmRdB7g8NuxP9xzn00= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP2S0u3x+CMOOaG2F5tb2vdbByndlfINEKpgkpH3o5XiNOc7UKoiytaEAS6e1c3qyt1H9wMGN4hL0DyTHKMhlJ0= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCCXjrNp0exN6HvNy3i+osWJ6TFiX6gN8XIIvSeLyo53AGUJdyJBLQI+csy8oiYwWOzcFQswOtx+Y8MocvONLug= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-21_09:54:56 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-21T10:13:30.855 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_b8ddeda347de6c45510a6df67440a1d298c37030/qa; will attempt to use it 2024-04-21T10:13:30.856 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_b8ddeda347de6c45510a6df67440a1d298c37030/qa/tasks 2024-04-21T10:13:30.856 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-21T10:13:30.858 INFO:teuthology.task.internal:Checking packages... 2024-04-21T10:13:30.885 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a630765c1f93c6e61d9b40c4b76437611d5ef29e' 2024-04-21T10:13:30.885 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-21T10:13:30.885 INFO:teuthology.packaging:ref: None 2024-04-21T10:13:30.885 INFO:teuthology.packaging:tag: None 2024-04-21T10:13:30.885 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-21T10:13:30.885 INFO:teuthology.packaging:sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T10:13:30.885 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-21T10:13:31.040 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3154.ga630765c 2024-04-21T10:13:31.042 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-21T10:13:31.049 INFO:teuthology.task.internal:no buildpackages task found 2024-04-21T10:13:31.049 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-21T10:13:31.056 INFO:teuthology.task.internal:Saving configuration 2024-04-21T10:13:31.080 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-21T10:13:31.087 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T10:13:31.112 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 10:08:12.875657', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ2lqbpudOBSVXMl416tiMvWri1zUhLjXS6Y/XKmRBkP4X8ME/S+lHPQhzNdM+J4GV9EfZmRdB7g8NuxP9xzn00='} 2024-04-21T10:13:31.135 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 10:08:12.880022', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP2S0u3x+CMOOaG2F5tb2vdbByndlfINEKpgkpH3o5XiNOc7UKoiytaEAS6e1c3qyt1H9wMGN4hL0DyTHKMhlJ0='} 2024-04-21T10:13:31.156 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 10:08:12.877501', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCCXjrNp0exN6HvNy3i+osWJ6TFiX6gN8XIIvSeLyo53AGUJdyJBLQI+csy8oiYwWOzcFQswOtx+Y8MocvONLug='} 2024-04-21T10:13:31.157 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-21T10:13:31.163 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-21T10:13:31.163 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-21T10:13:31.163 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-21T10:13:31.163 INFO:teuthology.run_tasks:Running task console_log... 2024-04-21T10:13:31.270 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcb6c7eb0d0>, signals=[15]) 2024-04-21T10:13:31.271 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-21T10:13:31.277 INFO:teuthology.task.internal:Opening connections... 2024-04-21T10:13:31.277 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-04-21T10:13:31.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T10:13:31.358 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-04-21T10:13:31.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T10:13:31.434 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-04-21T10:13:31.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T10:13:31.508 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-21T10:13:31.515 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-04-21T10:13:31.535 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-04-21T10:13:31.535 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-04-21T10:13:31.593 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2024-04-21T10:13:31.593 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="9" 2024-04-21T10:13:31.593 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2024-04-21T10:13:31.594 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2024-04-21T10:13:31.594 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="9" 2024-04-21T10:13:31.594 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el9" 2024-04-21T10:13:31.594 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T10:13:31.594 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-04-21T10:13:31.594 INFO:teuthology.orchestra.run.smithi026.stdout:LOGO="fedora-logo-icon" 2024-04-21T10:13:31.594 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T10:13:31.594 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2024-04-21T10:13:31.594 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T10:13:31.594 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T10:13:31.594 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T10:13:31.595 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-04-21T10:13:31.620 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-04-21T10:13:31.638 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-04-21T10:13:31.638 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-04-21T10:13:31.695 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-04-21T10:13:31.695 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-04-21T10:13:31.695 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-04-21T10:13:31.695 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-04-21T10:13:31.695 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-04-21T10:13:31.695 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-04-21T10:13:31.695 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T10:13:31.695 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-04-21T10:13:31.695 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-04-21T10:13:31.695 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T10:13:31.696 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-04-21T10:13:31.696 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T10:13:31.696 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T10:13:31.696 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T10:13:31.696 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-04-21T10:13:31.720 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-04-21T10:13:31.738 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-04-21T10:13:31.739 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-04-21T10:13:31.795 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-04-21T10:13:31.796 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2024-04-21T10:13:31.796 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-04-21T10:13:31.796 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-04-21T10:13:31.796 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2024-04-21T10:13:31.796 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2024-04-21T10:13:31.796 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T10:13:31.796 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-04-21T10:13:31.796 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2024-04-21T10:13:31.796 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T10:13:31.796 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-04-21T10:13:31.797 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T10:13:31.797 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T10:13:31.797 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T10:13:31.797 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-04-21T10:13:31.820 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-21T10:13:31.831 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-21T10:13:31.837 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-21T10:13:31.837 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-04-21T10:13:31.840 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-04-21T10:13:31.843 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-04-21T10:13:31.859 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-21T10:13:31.865 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-21T10:13:31.865 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-04-21T10:13:31.897 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-04-21T10:13:31.900 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-04-21T10:13:31.960 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-21T10:13:32.039 INFO:teuthology.run_tasks:Running task kernel... 2024-04-21T10:13:32.054 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-21T10:13:32.055 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-21T10:13:32.055 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-21T10:13:32.055 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T10:13:32.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:13:32.072 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-04-21T10:13:32.130 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-437.el9.x86_64 2024-04-21T10:13:32.131 INFO:teuthology.task.kernel:Running kernel on smithi026: 5.14.0-437.el9.x86_64 2024-04-21T10:13:32.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum install -y kernel 2024-04-21T10:13:36.207 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.0 MB 00:02 2024-04-21T10:13:37.982 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-21T10:13:54.481 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 495 kB/s | 6.2 MB 00:12 2024-04-21T10:13:55.915 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 16 kB 00:00 2024-04-21T10:13:57.311 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-21T10:14:02.525 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-21T10:14:04.490 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-21T10:14:04.490 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-21T10:14:04.579 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-04-21T10:14:04.580 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-04-21T10:14:04.580 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-04-21T10:14:04.680 DEBUG:teuthology.orchestra.run.smithi026:> echo no | sudo yum reinstall kernel || true 2024-04-21T10:14:05.199 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:03 ago on Sun 21 Apr 2024 10:14:02 AM UTC. 2024-04-21T10:14:05.323 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-04-21T10:14:05.336 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-04-21T10:14:05.336 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-04-21T10:14:05.336 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-04-21T10:14:05.336 INFO:teuthology.orchestra.run.smithi026.stdout:Reinstalling: 2024-04-21T10:14:05.336 INFO:teuthology.orchestra.run.smithi026.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-21T10:14:05.336 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:14:05.336 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-04-21T10:14:05.336 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-04-21T10:14:05.336 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:14:05.337 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 255 k 2024-04-21T10:14:05.337 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 0 2024-04-21T10:14:05.337 INFO:teuthology.orchestra.run.smithi026.stderr:Operation aborted. 2024-04-21T10:14:05.388 INFO:teuthology.orchestra.run.smithi026.stdout:Is this ok [y/N]: 2024-04-21T10:14:05.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum reinstall -y kernel || true 2024-04-21T10:14:05.903 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:03 ago on Sun 21 Apr 2024 10:14:02 AM UTC. 2024-04-21T10:14:06.025 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-04-21T10:14:06.025 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-04-21T10:14:06.026 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-04-21T10:14:06.026 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-04-21T10:14:06.026 INFO:teuthology.orchestra.run.smithi026.stdout:Reinstalling: 2024-04-21T10:14:06.026 INFO:teuthology.orchestra.run.smithi026.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-21T10:14:06.026 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:14:06.026 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-04-21T10:14:06.026 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-04-21T10:14:06.026 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:14:06.026 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 255 k 2024-04-21T10:14:06.027 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 0 2024-04-21T10:14:06.027 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-04-21T10:14:06.319 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.4 MB/s | 255 kB 00:00 2024-04-21T10:14:06.319 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-04-21T10:14:06.319 INFO:teuthology.orchestra.run.smithi026.stdout:Total 870 kB/s | 255 kB 00:00 2024-04-21T10:14:06.337 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-04-21T10:14:06.529 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-04-21T10:14:06.529 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-04-21T10:14:06.534 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-04-21T10:14:06.534 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-04-21T10:14:06.665 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-04-21T10:14:06.728 INFO:teuthology.orchestra.run.smithi026.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-21T10:14:06.728 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T10:14:07.346 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T10:14:07.346 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-21T10:14:08.053 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T10:14:08.053 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:14:08.053 INFO:teuthology.orchestra.run.smithi026.stdout:Reinstalled: 2024-04-21T10:14:08.053 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-21T10:14:08.053 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:14:08.053 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-04-21T10:14:08.223 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-04-21T10:14:08.276 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-21T10:14:08.276 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-21T10:14:08.276 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-21T10:14:08.277 DEBUG:teuthology.parallel:result is None 2024-04-21T10:14:08.277 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-21T10:14:08.392 INFO:teuthology.task.internal:Creating test directory... 2024-04-21T10:14:08.393 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T10:14:08.397 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T10:14:08.400 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T10:14:08.418 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-21T10:14:08.435 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-21T10:14:08.444 INFO:teuthology.task.internal:Creating archive directory... 2024-04-21T10:14:08.444 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T10:14:08.452 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T10:14:08.460 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T10:14:08.563 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-21T10:14:08.570 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-21T10:14:08.570 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T10:14:08.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:14:08.587 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T10:14:08.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:14:08.603 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T10:14:08.605 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T10:14:08.630 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T10:14:08.631 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T10:14:08.664 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T10:14:08.667 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T10:14:08.701 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T10:14:08.707 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T10:14:08.718 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T10:14:08.720 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-21T10:14:08.728 INFO:teuthology.task.internal:Configuring sudo... 2024-04-21T10:14:08.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T10:14:08.731 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T10:14:08.752 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T10:14:08.779 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-21T10:14:08.790 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-21T10:14:08.790 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T10:14:08.796 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T10:14:08.820 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T10:14:08.836 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T10:14:08.891 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T10:14:08.941 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T10:14:09.022 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T10:14:09.063 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:14:09.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T10:14:09.127 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T10:14:09.172 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T10:14:09.223 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T10:14:09.305 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T10:14:09.349 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-21T10:14:09.349 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T10:14:09.414 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T10:14:09.459 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T10:14:09.521 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T10:14:09.564 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T10:14:09.606 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-21T10:14:09.606 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T10:14:09.671 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-04-21T10:14:09.673 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-04-21T10:14:09.676 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-04-21T10:14:09.705 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T10:14:09.709 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T10:14:09.743 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T10:14:10.223 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-21T10:14:10.231 INFO:teuthology.task.internal:Starting timer... 2024-04-21T10:14:10.231 INFO:teuthology.run_tasks:Running task pcp... 2024-04-21T10:14:10.256 INFO:teuthology.run_tasks:Running task selinux... 2024-04-21T10:14:10.264 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-21T10:14:10.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-04-21T10:14:10.340 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-04-21T10:14:10.342 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-04-21T10:14:10.399 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-04-21T10:14:10.401 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-04-21T10:14:10.463 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-04-21T10:14:10.466 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-21T10:14:10.466 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-04-21T10:14:10.494 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-04-21T10:14:10.494 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-04-21T10:14:10.514 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-04-21T10:14:10.515 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-04-21T10:14:10.621 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-04-21T10:14:10.621 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2024-04-21T10:14:10.621 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T10:14:10.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:14:10.649 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T10:14:10.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:14:10.675 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T10:14:10.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:14:10.702 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-21T10:14:10.702 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-04-21T10:14:10.728 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-04-21T10:14:10.753 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-04-21T10:14:10.790 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-21T10:14:10.801 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-21T10:14:10.806 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-21T10:14:10.934 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-21T10:14:10.950 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-21T10:14:10.951 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi169.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-21T10:18:11.450 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-04-21T10:18:11.452 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-04-21T10:18:11.453 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T10:18:11.548 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-04-21T10:18:11.627 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-04-21T10:18:11.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-21T10:18:11.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T10:18:11.722 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-21T10:18:11.814 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-21T10:18:11.814 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-21T10:18:11.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T10:18:11.890 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-04-21T10:18:11.970 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-21T10:18:11.970 INFO:teuthology.run_tasks:Running task clock... 2024-04-21T10:18:11.982 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-21T10:18:11.983 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T10:18:11.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T10:18:11.986 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T10:18:11.986 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T10:18:11.988 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T10:18:11.988 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T10:18:12.014 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T10:18:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T10:18:12.030 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T10:18:12.039 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T10:18:12.040 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T10:18:12.053 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T10:18:12.055 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-04-21T10:18:12.064 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-04-21T10:18:12.067 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-04-21T10:18:12.074 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-04-21T10:18:12.076 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-04-21T10:18:12.081 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T10:18:12.087 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-04-21T10:18:12.088 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T10:18:12.095 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T10:18:12.101 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T10:18:12.102 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T10:18:12.113 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T10:18:12.150 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-04-21T10:18:12.153 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T10:18:12.153 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-04-21T10:18:12.153 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:18:12.153 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:18:12.153 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:18:12.153 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:18:12.165 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-21T10:18:12.166 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-04-21T10:18:12.168 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T10:18:12.168 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-21T10:18:12.168 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:18:12.168 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:18:12.168 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:18:12.168 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:18:12.169 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T10:18:12.169 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-04-21T10:18:12.169 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:18:12.169 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:18:12.169 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:18:12.169 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:18:12.170 INFO:teuthology.run_tasks:Running task install... 2024-04-21T10:18:12.179 DEBUG:teuthology.task.install:project ceph 2024-04-21T10:18:12.179 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e'}} 2024-04-21T10:18:12.179 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e'} 2024-04-21T10:18:12.179 INFO:teuthology.task.install:Using flavor: default 2024-04-21T10:18:12.185 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-21T10:18:12.185 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-21T10:18:12.185 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T10:18:12.186 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T10:18:12.188 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T10:18:12.188 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T10:18:12.189 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T10:18:12.189 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T10:18:12.409 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T10:18:12.409 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T10:18:12.413 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T10:18:12.413 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T10:18:12.422 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T10:18:12.422 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T10:18:12.537 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T10:18:12.537 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-21T10:18:12.537 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T10:18:12.563 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T10:18:12.563 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T10:18:12.591 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T10:18:12.591 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-21T10:18:12.591 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T10:18:12.598 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T10:18:12.598 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:18:12.598 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T10:18:12.616 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T10:18:12.616 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T10:18:12.626 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T10:18:12.627 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T10:18:12.628 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T10:18:12.681 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T10:18:12.694 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T10:18:12.701 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T10:18:12.754 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T10:18:12.764 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-04-21T10:18:12.766 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-04-21T10:18:12.771 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T10:18:12.817 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2024-04-21T10:18:12.819 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2024-04-21T10:18:12.836 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-04-21T10:18:12.838 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-04-21T10:18:12.992 INFO:teuthology.orchestra.run.smithi169.stdout:48 files removed 2024-04-21T10:18:13.020 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T10:18:13.046 INFO:teuthology.orchestra.run.smithi190.stdout:48 files removed 2024-04-21T10:18:13.073 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T10:18:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:48 files removed 2024-04-21T10:18:13.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T10:18:14.073 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 105 kB/s | 83 kB 00:00 2024-04-21T10:18:14.156 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 101 kB/s | 83 kB 00:00 2024-04-21T10:18:14.283 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 92 kB/s | 83 kB 00:00 2024-04-21T10:18:14.590 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-21T10:18:14.649 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-04-21T10:18:14.747 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-04-21T10:18:15.032 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-04-21T10:18:15.116 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-21T10:18:15.139 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-04-21T10:18:15.708 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-21T10:18:15.781 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-21T10:18:17.867 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-21T10:18:18.278 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-21T10:18:22.499 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.2 MB 00:00 2024-04-21T10:18:23.535 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-04-21T10:18:23.965 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 926 kB/s | 8.0 MB 00:08 2024-04-21T10:18:24.334 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-21T10:18:25.194 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-21T10:18:25.668 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-21T10:18:27.303 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:01 2024-04-21T10:18:30.876 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-21T10:18:32.361 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-21T10:18:33.137 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T10:18:33.138 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T10:18:33.149 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T10:18:33.150 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T10:18:33.150 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T10:18:33.151 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T10:18:33.153 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T10:18:33.155 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T10:18:33.156 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T10:18:33.221 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-04-21T10:18:33.223 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 2.3 MB/s | 19 MB 00:08 2024-04-21T10:18:33.230 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-04-21T10:18:33.230 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-04-21T10:18:33.230 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-04-21T10:18:33.230 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T10:18:33.231 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T10:18:33.232 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T10:18:33.233 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T10:18:33.234 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T10:18:33.235 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T10:18:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T10:18:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T10:18:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T10:18:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T10:18:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T10:18:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T10:18:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T10:18:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T10:18:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T10:18:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T10:18:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T10:18:33.236 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T10:18:33.237 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T10:18:33.238 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T10:18:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T10:18:33.240 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T10:18:33.241 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T10:18:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T10:18:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T10:18:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T10:18:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T10:18:33.242 INFO:teuthology.orchestra.run.smithi026.stdout:Installing weak dependencies: 2024-04-21T10:18:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T10:18:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:18:33.242 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-04-21T10:18:33.242 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-04-21T10:18:33.242 INFO:teuthology.orchestra.run.smithi026.stdout:Install 148 Packages 2024-04-21T10:18:33.242 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 2 Packages 2024-04-21T10:18:33.242 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:18:33.243 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 218 M 2024-04-21T10:18:33.243 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-04-21T10:18:34.336 INFO:teuthology.orchestra.run.smithi026.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 32 kB/s | 6.6 kB 00:00 2024-04-21T10:18:34.521 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T10:18:34.522 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T10:18:34.532 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T10:18:34.533 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T10:18:34.534 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T10:18:34.535 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T10:18:34.536 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T10:18:34.539 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T10:18:34.540 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T10:18:34.552 INFO:teuthology.orchestra.run.smithi026.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 4.0 MB/s | 890 kB 00:00 2024-04-21T10:18:34.600 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-21T10:18:34.618 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-21T10:18:34.619 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-04-21T10:18:34.619 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-21T10:18:34.619 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-04-21T10:18:34.619 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T10:18:34.619 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T10:18:34.619 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T10:18:34.619 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T10:18:34.619 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T10:18:34.619 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T10:18:34.619 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T10:18:34.619 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T10:18:34.619 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T10:18:34.620 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T10:18:34.621 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T10:18:34.622 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-04-21T10:18:34.622 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T10:18:34.622 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T10:18:34.622 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-04-21T10:18:34.622 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T10:18:34.622 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T10:18:34.622 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T10:18:34.622 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T10:18:34.622 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T10:18:34.622 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T10:18:34.622 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T10:18:34.622 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T10:18:34.623 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T10:18:34.623 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T10:18:34.623 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T10:18:34.623 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T10:18:34.623 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T10:18:34.623 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T10:18:34.623 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T10:18:34.623 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T10:18:34.623 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T10:18:34.623 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T10:18:34.623 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T10:18:34.624 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T10:18:34.624 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T10:18:34.624 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T10:18:34.624 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T10:18:34.624 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T10:18:34.624 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T10:18:34.624 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T10:18:34.624 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T10:18:34.624 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T10:18:34.625 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T10:18:34.626 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T10:18:34.626 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T10:18:34.626 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T10:18:34.626 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T10:18:34.626 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T10:18:34.626 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T10:18:34.626 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T10:18:34.626 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T10:18:34.627 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T10:18:34.628 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T10:18:34.629 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T10:18:34.630 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T10:18:34.630 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T10:18:34.630 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T10:18:34.630 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T10:18:34.630 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T10:18:34.630 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T10:18:34.630 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T10:18:34.630 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T10:18:34.631 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T10:18:34.631 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T10:18:34.631 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T10:18:34.631 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T10:18:34.631 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T10:18:34.631 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T10:18:34.632 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T10:18:34.632 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T10:18:34.632 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T10:18:34.632 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-04-21T10:18:34.632 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T10:18:34.633 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:18:34.633 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-21T10:18:34.633 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-21T10:18:34.633 INFO:teuthology.orchestra.run.smithi169.stdout:Install 148 Packages 2024-04-21T10:18:34.633 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 2 Packages 2024-04-21T10:18:34.633 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:18:34.634 INFO:teuthology.orchestra.run.smithi026.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.9 MB/s | 145 kB 00:00 2024-04-21T10:18:34.634 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 218 M 2024-04-21T10:18:34.634 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-04-21T10:18:34.920 INFO:teuthology.orchestra.run.smithi026.stdout:(4/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 6.8 MB/s | 5.4 MB 00:00 2024-04-21T10:18:35.003 INFO:teuthology.orchestra.run.smithi026.stdout:(5/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 6.3 MB/s | 2.4 MB 00:00 2024-04-21T10:18:35.112 INFO:teuthology.orchestra.run.smithi026.stdout:(6/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-04-21T10:18:35.529 INFO:teuthology.orchestra.run.smithi169.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-21T10:18:35.695 INFO:teuthology.orchestra.run.smithi026.stdout:(7/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 6.8 MB/s | 4.7 MB 00:00 2024-04-21T10:18:36.487 INFO:teuthology.orchestra.run.smithi026.stdout:(8/150): ceph-common-19.0.0-3154.ga630765c.el9. 9.0 MB/s | 21 MB 00:02 2024-04-21T10:18:36.513 INFO:teuthology.orchestra.run.smithi169.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 905 kB/s | 890 kB 00:00 2024-04-21T10:18:36.555 INFO:teuthology.orchestra.run.smithi026.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 378 kB/s | 25 kB 00:00 2024-04-21T10:18:36.697 INFO:teuthology.orchestra.run.smithi169.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 792 kB/s | 145 kB 00:00 2024-04-21T10:18:36.880 INFO:teuthology.orchestra.run.smithi026.stdout:(10/150): ceph-osd-19.0.0-3154.ga630765c.el9.x8 9.5 MB/s | 17 MB 00:01 2024-04-21T10:18:36.963 INFO:teuthology.orchestra.run.smithi026.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 2.6 MB/s | 225 kB 00:00 2024-04-21T10:18:37.047 INFO:teuthology.orchestra.run.smithi026.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 325 kB/s | 27 kB 00:00 2024-04-21T10:18:37.122 INFO:teuthology.orchestra.run.smithi026.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 422 kB/s | 32 kB 00:00 2024-04-21T10:18:37.231 INFO:teuthology.orchestra.run.smithi026.stdout:(14/150): ceph-radosgw-19.0.0-3154.ga630765c.el 6.9 MB/s | 11 MB 00:01 2024-04-21T10:18:37.273 INFO:teuthology.orchestra.run.smithi026.stdout:(15/150): libcephfs2-19.0.0-3154.ga630765c.el9. 4.9 MB/s | 746 kB 00:00 2024-04-21T10:18:37.306 INFO:teuthology.orchestra.run.smithi026.stdout:(16/150): libcephsqlite-19.0.0-3154.ga630765c.e 2.1 MB/s | 163 kB 00:00 2024-04-21T10:18:37.348 INFO:teuthology.orchestra.run.smithi026.stdout:(17/150): librados-devel-19.0.0-3154.ga630765c. 1.6 MB/s | 127 kB 00:00 2024-04-21T10:18:37.398 INFO:teuthology.orchestra.run.smithi026.stdout:(18/150): libradosstriper1-19.0.0-3154.ga630765 5.3 MB/s | 501 kB 00:00 2024-04-21T10:18:37.472 INFO:teuthology.orchestra.run.smithi169.stdout:(4/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-04-21T10:18:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 541 kB/s | 45 kB 00:00 2024-04-21T10:18:37.582 INFO:teuthology.orchestra.run.smithi026.stdout:(20/150): python3-ceph-common-19.0.0-3154.ga630 1.3 MB/s | 135 kB 00:00 2024-04-21T10:18:37.682 INFO:teuthology.orchestra.run.smithi026.stdout:(21/150): python3-cephfs-19.0.0-3154.ga630765c. 1.6 MB/s | 162 kB 00:00 2024-04-21T10:18:37.743 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 6.7 MB/s | 6.2 MB 00:00 2024-04-21T10:18:37.880 INFO:teuthology.orchestra.run.smithi169.stdout:(5/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-21T10:18:37.983 INFO:teuthology.orchestra.run.smithi026.stdout:(22/150): librgw2-19.0.0-3154.ga630765c.el9.x86 8.5 MB/s | 5.4 MB 00:00 2024-04-21T10:18:38.022 INFO:teuthology.orchestra.run.smithi169.stdout:(6/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 1.8 MB/s | 2.4 MB 00:01 2024-04-21T10:18:38.059 INFO:teuthology.orchestra.run.smithi026.stdout:(23/150): python3-rbd-19.0.0-3154.ga630765c.el9 3.9 MB/s | 298 kB 00:00 2024-04-21T10:18:38.167 INFO:teuthology.orchestra.run.smithi026.stdout:(24/150): python3-rgw-19.0.0-3154.ga630765c.el9 920 kB/s | 100 kB 00:00 2024-04-21T10:18:38.192 INFO:teuthology.orchestra.run.smithi026.stdout:(25/150): python3-rados-19.0.0-3154.ga630765c.e 630 kB/s | 321 kB 00:00 2024-04-21T10:18:38.606 INFO:teuthology.orchestra.run.smithi169.stdout:(7/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 6.5 MB/s | 4.7 MB 00:00 2024-04-21T10:18:38.768 INFO:teuthology.orchestra.run.smithi026.stdout:(26/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 142 kB/s | 85 kB 00:00 2024-04-21T10:18:38.835 INFO:teuthology.orchestra.run.smithi026.stdout:(27/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-21T10:18:38.902 INFO:teuthology.orchestra.run.smithi026.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3154.g 512 kB/s | 34 kB 00:00 2024-04-21T10:18:38.977 INFO:teuthology.orchestra.run.smithi026.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 1.9 MB/s | 142 kB 00:00 2024-04-21T10:18:39.175 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-21T10:18:39.585 INFO:teuthology.orchestra.run.smithi026.stdout:(30/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 5.8 MB/s | 3.5 MB 00:00 2024-04-21T10:18:39.711 INFO:teuthology.orchestra.run.smithi026.stdout:(31/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 2.0 MB/s | 3.1 MB 00:01 2024-04-21T10:18:39.861 INFO:teuthology.orchestra.run.smithi026.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 1.6 MB/s | 249 kB 00:00 2024-04-21T10:18:39.919 INFO:teuthology.orchestra.run.smithi026.stdout:(33/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 848 kB/s | 49 kB 00:00 2024-04-21T10:18:39.986 INFO:teuthology.orchestra.run.smithi026.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3154.ga 247 kB/s | 16 kB 00:00 2024-04-21T10:18:40.022 INFO:teuthology.orchestra.run.smithi169.stdout:(8/150): ceph-radosgw-19.0.0-3154.ga630765c.el9 7.5 MB/s | 11 MB 00:01 2024-04-21T10:18:40.106 INFO:teuthology.orchestra.run.smithi169.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 303 kB/s | 25 kB 00:00 2024-04-21T10:18:40.145 INFO:teuthology.orchestra.run.smithi026.stdout:(35/150): ceph-volume-19.0.0-3154.ga630765c.el9 1.7 MB/s | 273 kB 00:00 2024-04-21T10:18:40.412 INFO:teuthology.orchestra.run.smithi026.stdout:(36/150): cephadm-19.0.0-3154.ga630765c.el9.noa 2.8 MB/s | 762 kB 00:00 2024-04-21T10:18:40.496 INFO:teuthology.orchestra.run.smithi026.stdout:(37/150): cephfs-top-19.0.0-3154.ga630765c.el9. 219 kB/s | 18 kB 00:00 2024-04-21T10:18:40.618 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-21T10:18:40.638 INFO:teuthology.orchestra.run.smithi026.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-04-21T10:18:40.730 INFO:teuthology.orchestra.run.smithi026.stdout:(39/150): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-04-21T10:18:40.805 INFO:teuthology.orchestra.run.smithi026.stdout:(40/150): libgfortran-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 803 kB 00:00 2024-04-21T10:18:40.847 INFO:teuthology.orchestra.run.smithi026.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-04-21T10:18:40.872 INFO:teuthology.orchestra.run.smithi026.stdout:(42/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-21T10:18:40.914 INFO:teuthology.orchestra.run.smithi026.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-21T10:18:40.939 INFO:teuthology.orchestra.run.smithi026.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-21T10:18:40.964 INFO:teuthology.orchestra.run.smithi026.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-21T10:18:40.990 INFO:teuthology.orchestra.run.smithi026.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-21T10:18:41.015 INFO:teuthology.orchestra.run.smithi026.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-21T10:18:41.790 INFO:teuthology.orchestra.run.smithi026.stdout:(48/150): ceph-test-19.0.0-3154.ga630765c.el9.x 9.4 MB/s | 49 MB 00:05 2024-04-21T10:18:41.824 INFO:teuthology.orchestra.run.smithi026.stdout:(49/150): automake-1.16.2-8.el9.noarch.rpm 826 kB/s | 689 kB 00:00 2024-04-21T10:18:41.908 INFO:teuthology.orchestra.run.smithi026.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 259 kB/s | 21 kB 00:00 2024-04-21T10:18:42.058 INFO:teuthology.orchestra.run.smithi026.stdout:(51/150): bison-3.7.4-5.el9.x86_64.rpm 905 kB/s | 943 kB 00:01 2024-04-21T10:18:42.083 INFO:teuthology.orchestra.run.smithi026.stdout:(52/150): boost-program-options-1.75.0-8.el9.x8 366 kB/s | 107 kB 00:00 2024-04-21T10:18:42.117 INFO:teuthology.orchestra.run.smithi026.stdout:(53/150): flex-2.6.4-9.el9.x86_64.rpm 1.5 MB/s | 313 kB 00:00 2024-04-21T10:18:42.168 INFO:teuthology.orchestra.run.smithi026.stdout:(54/150): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-04-21T10:18:42.201 INFO:teuthology.orchestra.run.smithi026.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-04-21T10:18:42.251 INFO:teuthology.orchestra.run.smithi026.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-04-21T10:18:42.334 INFO:teuthology.orchestra.run.smithi026.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 544 kB/s | 45 kB 00:00 2024-04-21T10:18:42.351 INFO:teuthology.orchestra.run.smithi026.stdout:(58/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-04-21T10:18:42.668 INFO:teuthology.orchestra.run.smithi026.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 1.8 MB/s | 582 kB 00:00 2024-04-21T10:18:42.744 INFO:teuthology.orchestra.run.smithi026.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 1.6 MB/s | 662 kB 00:00 2024-04-21T10:18:42.769 INFO:teuthology.orchestra.run.smithi026.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-04-21T10:18:42.794 INFO:teuthology.orchestra.run.smithi026.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-21T10:18:42.878 INFO:teuthology.orchestra.run.smithi026.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-21T10:18:42.903 INFO:teuthology.orchestra.run.smithi026.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-21T10:18:42.928 INFO:teuthology.orchestra.run.smithi026.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-04-21T10:18:43.229 INFO:teuthology.orchestra.run.smithi026.stdout:(66/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-04-21T10:18:43.372 INFO:teuthology.orchestra.run.smithi169.stdout:(10/150): ceph-test-19.0.0-3154.ga630765c.el9.x 15 MB/s | 49 MB 00:03 2024-04-21T10:18:43.441 INFO:teuthology.orchestra.run.smithi169.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 3.2 MB/s | 225 kB 00:00 2024-04-21T10:18:43.492 INFO:teuthology.orchestra.run.smithi169.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 543 kB/s | 27 kB 00:00 2024-04-21T10:18:43.534 INFO:teuthology.orchestra.run.smithi169.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 755 kB/s | 32 kB 00:00 2024-04-21T10:18:43.618 INFO:teuthology.orchestra.run.smithi169.stdout:(14/150): libcephfs2-19.0.0-3154.ga630765c.el9. 8.8 MB/s | 746 kB 00:00 2024-04-21T10:18:43.696 INFO:teuthology.orchestra.run.smithi026.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-04-21T10:18:43.785 INFO:teuthology.orchestra.run.smithi169.stdout:(15/150): ceph-osd-19.0.0-3154.ga630765c.el9.x8 2.9 MB/s | 17 MB 00:05 2024-04-21T10:18:43.810 INFO:teuthology.orchestra.run.smithi169.stdout:(16/150): libcephsqlite-19.0.0-3154.ga630765c.e 851 kB/s | 163 kB 00:00 2024-04-21T10:18:43.869 INFO:teuthology.orchestra.run.smithi169.stdout:(17/150): libradosstriper1-19.0.0-3154.ga630765 8.4 MB/s | 501 kB 00:00 2024-04-21T10:18:43.885 INFO:teuthology.orchestra.run.smithi169.stdout:(18/150): librados-devel-19.0.0-3154.ga630765c. 1.2 MB/s | 127 kB 00:00 2024-04-21T10:18:43.887 INFO:teuthology.orchestra.run.smithi026.stdout:(68/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 4.3 MB/s | 4.1 MB 00:00 2024-04-21T10:18:43.980 INFO:teuthology.orchestra.run.smithi026.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-21T10:18:43.994 INFO:teuthology.orchestra.run.smithi169.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 416 kB/s | 45 kB 00:00 2024-04-21T10:18:44.086 INFO:teuthology.orchestra.run.smithi169.stdout:(20/150): python3-ceph-common-19.0.0-3154.ga630 1.4 MB/s | 135 kB 00:00 2024-04-21T10:18:44.113 INFO:teuthology.orchestra.run.smithi026.stdout:(70/150): postgresql-contrib-13.14-1.el9.x86_64 2.1 MB/s | 911 kB 00:00 2024-04-21T10:18:44.211 INFO:teuthology.orchestra.run.smithi169.stdout:(21/150): librgw2-19.0.0-3154.ga630765c.el9.x86 16 MB/s | 5.4 MB 00:00 2024-04-21T10:18:44.237 INFO:teuthology.orchestra.run.smithi169.stdout:(22/150): python3-cephfs-19.0.0-3154.ga630765c. 1.1 MB/s | 162 kB 00:00 2024-04-21T10:18:44.262 INFO:teuthology.orchestra.run.smithi169.stdout:(23/150): python3-rados-19.0.0-3154.ga630765c.e 6.3 MB/s | 321 kB 00:00 2024-04-21T10:18:44.295 INFO:teuthology.orchestra.run.smithi169.stdout:(24/150): python3-rgw-19.0.0-3154.ga630765c.el9 2.9 MB/s | 100 kB 00:00 2024-04-21T10:18:44.321 INFO:teuthology.orchestra.run.smithi026.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-04-21T10:18:44.337 INFO:teuthology.orchestra.run.smithi169.stdout:(25/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 2.0 MB/s | 85 kB 00:00 2024-04-21T10:18:44.363 INFO:teuthology.orchestra.run.smithi169.stdout:(26/150): python3-rbd-19.0.0-3154.ga630765c.el9 2.3 MB/s | 298 kB 00:00 2024-04-21T10:18:44.380 INFO:teuthology.orchestra.run.smithi026.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-21T10:18:44.446 INFO:teuthology.orchestra.run.smithi169.stdout:(27/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-21T10:18:44.522 INFO:teuthology.orchestra.run.smithi169.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3154.g 457 kB/s | 34 kB 00:00 2024-04-21T10:18:44.589 INFO:teuthology.orchestra.run.smithi169.stdout:(29/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 12 MB/s | 3.1 MB 00:00 2024-04-21T10:18:44.605 INFO:teuthology.orchestra.run.smithi169.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 1.7 MB/s | 142 kB 00:00 2024-04-21T10:18:44.697 INFO:teuthology.orchestra.run.smithi026.stdout:(73/150): postgresql-server-13.14-1.el9.x86_64. 8.1 MB/s | 5.8 MB 00:00 2024-04-21T10:18:44.764 INFO:teuthology.orchestra.run.smithi026.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-21T10:18:44.823 INFO:teuthology.orchestra.run.smithi169.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 15 MB/s | 3.5 MB 00:00 2024-04-21T10:18:44.831 INFO:teuthology.orchestra.run.smithi026.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-21T10:18:44.873 INFO:teuthology.orchestra.run.smithi169.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 4.9 MB/s | 249 kB 00:00 2024-04-21T10:18:44.881 INFO:teuthology.orchestra.run.smithi026.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-21T10:18:44.923 INFO:teuthology.orchestra.run.smithi169.stdout:(33/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 991 kB/s | 49 kB 00:00 2024-04-21T10:18:44.931 INFO:teuthology.orchestra.run.smithi026.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-21T10:18:44.965 INFO:teuthology.orchestra.run.smithi169.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3154.ga 396 kB/s | 16 kB 00:00 2024-04-21T10:18:45.015 INFO:teuthology.orchestra.run.smithi169.stdout:(35/150): ceph-volume-19.0.0-3154.ga630765c.el9 5.3 MB/s | 273 kB 00:00 2024-04-21T10:18:45.031 INFO:teuthology.orchestra.run.smithi026.stdout:(78/150): openblas-openmp-0.3.21-2.el9.x86_64.r 2.2 MB/s | 4.7 MB 00:02 2024-04-21T10:18:45.057 INFO:teuthology.orchestra.run.smithi026.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-21T10:18:45.074 INFO:teuthology.orchestra.run.smithi026.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 827 kB/s | 35 kB 00:00 2024-04-21T10:18:45.099 INFO:teuthology.orchestra.run.smithi169.stdout:(36/150): cephadm-19.0.0-3154.ga630765c.el9.noa 9.0 MB/s | 762 kB 00:00 2024-04-21T10:18:45.149 INFO:teuthology.orchestra.run.smithi169.stdout:(37/150): cephfs-top-19.0.0-3154.ga630765c.el9. 364 kB/s | 18 kB 00:00 2024-04-21T10:18:45.174 INFO:teuthology.orchestra.run.smithi026.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-21T10:18:45.308 INFO:teuthology.orchestra.run.smithi026.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-04-21T10:18:45.383 INFO:teuthology.orchestra.run.smithi026.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-21T10:18:45.458 INFO:teuthology.orchestra.run.smithi026.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 564 kB/s | 42 kB 00:00 2024-04-21T10:18:45.575 INFO:teuthology.orchestra.run.smithi026.stdout:(85/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.8 MB/s | 5.1 MB 00:00 2024-04-21T10:18:45.609 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-21T10:18:45.617 INFO:teuthology.orchestra.run.smithi026.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-04-21T10:18:45.642 INFO:teuthology.orchestra.run.smithi026.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-21T10:18:45.693 INFO:teuthology.orchestra.run.smithi026.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-21T10:18:45.718 INFO:teuthology.orchestra.run.smithi026.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-04-21T10:18:45.743 INFO:teuthology.orchestra.run.smithi026.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-21T10:18:45.793 INFO:teuthology.orchestra.run.smithi026.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-04-21T10:18:45.852 INFO:teuthology.orchestra.run.smithi026.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 760 kB/s | 44 kB 00:00 2024-04-21T10:18:45.910 INFO:teuthology.orchestra.run.smithi026.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-21T10:18:45.933 INFO:teuthology.orchestra.run.smithi169.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 102 kB/s | 80 kB 00:00 2024-04-21T10:18:45.986 INFO:teuthology.orchestra.run.smithi026.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 477 kB/s | 36 kB 00:00 2024-04-21T10:18:46.045 INFO:teuthology.orchestra.run.smithi026.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-04-21T10:18:46.103 INFO:teuthology.orchestra.run.smithi026.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-21T10:18:46.179 INFO:teuthology.orchestra.run.smithi026.stdout:(97/150): python3-babel-2.9.1-2.el9.noarch.rpm 3.3 MB/s | 6.0 MB 00:01 2024-04-21T10:18:46.196 INFO:teuthology.orchestra.run.smithi026.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 628 kB/s | 58 kB 00:00 2024-04-21T10:18:46.238 INFO:teuthology.orchestra.run.smithi026.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-21T10:18:46.263 INFO:teuthology.orchestra.run.smithi026.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 951 kB/s | 64 kB 00:00 2024-04-21T10:18:46.280 INFO:teuthology.orchestra.run.smithi026.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-04-21T10:18:46.388 INFO:teuthology.orchestra.run.smithi026.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-04-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi169.stdout:(39/150): ceph-mgr-diskprediction-local-19.0.0- 4.1 MB/s | 7.4 MB 00:01 2024-04-21T10:18:46.514 INFO:teuthology.orchestra.run.smithi026.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-04-21T10:18:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-21T10:18:46.885 INFO:teuthology.orchestra.run.smithi169.stdout:(40/150): ceph-common-19.0.0-3154.ga630765c.el9 1.8 MB/s | 21 MB 00:11 2024-04-21T10:18:46.931 INFO:teuthology.orchestra.run.smithi026.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 741 kB/s | 309 kB 00:00 2024-04-21T10:18:46.956 INFO:teuthology.orchestra.run.smithi026.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-04-21T10:18:47.156 INFO:teuthology.orchestra.run.smithi026.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 130 kB/s | 26 kB 00:00 2024-04-21T10:18:47.240 INFO:teuthology.orchestra.run.smithi026.stdout:(108/150): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-21T10:18:47.266 INFO:teuthology.orchestra.run.smithi026.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 456 kB/s | 49 kB 00:00 2024-04-21T10:18:47.324 INFO:teuthology.orchestra.run.smithi026.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 809 kB/s | 67 kB 00:00 2024-04-21T10:18:47.341 INFO:teuthology.orchestra.run.smithi026.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-21T10:18:47.450 INFO:teuthology.orchestra.run.smithi026.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 838 kB 00:00 2024-04-21T10:18:47.467 INFO:teuthology.orchestra.run.smithi169.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 189 kB/s | 195 kB 00:01 2024-04-21T10:18:47.475 INFO:teuthology.orchestra.run.smithi026.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-04-21T10:18:47.525 INFO:teuthology.orchestra.run.smithi026.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 587 kB/s | 43 kB 00:00 2024-04-21T10:18:47.550 INFO:teuthology.orchestra.run.smithi026.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 429 kB/s | 32 kB 00:00 2024-04-21T10:18:47.634 INFO:teuthology.orchestra.run.smithi026.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-21T10:18:47.659 INFO:teuthology.orchestra.run.smithi026.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-21T10:18:47.726 INFO:teuthology.orchestra.run.smithi026.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-21T10:18:47.743 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T10:18:47.744 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T10:18:47.750 INFO:teuthology.orchestra.run.smithi169.stdout:(42/150): libgfortran-11.4.1-3.el9.x86_64.rpm 442 kB/s | 803 kB 00:01 2024-04-21T10:18:47.751 INFO:teuthology.orchestra.run.smithi026.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-21T10:18:47.754 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T10:18:47.756 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T10:18:47.756 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T10:18:47.757 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T10:18:47.758 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T10:18:47.760 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T10:18:47.761 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T10:18:47.810 INFO:teuthology.orchestra.run.smithi026.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 426 kB/s | 35 kB 00:00 2024-04-21T10:18:47.821 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-04-21T10:18:47.844 INFO:teuthology.orchestra.run.smithi026.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-04-21T10:18:47.855 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-04-21T10:18:47.855 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-04-21T10:18:47.855 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-04-21T10:18:47.855 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-04-21T10:18:47.855 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T10:18:47.855 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T10:18:47.855 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T10:18:47.855 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T10:18:47.855 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T10:18:47.855 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T10:18:47.855 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T10:18:47.856 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T10:18:47.857 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T10:18:47.858 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T10:18:47.859 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T10:18:47.859 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T10:18:47.859 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T10:18:47.859 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T10:18:47.859 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T10:18:47.859 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T10:18:47.859 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T10:18:47.859 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T10:18:47.860 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T10:18:47.861 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T10:18:47.862 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T10:18:47.862 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T10:18:47.862 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T10:18:47.862 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T10:18:47.862 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T10:18:47.862 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T10:18:47.862 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T10:18:47.862 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T10:18:47.862 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T10:18:47.862 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T10:18:47.862 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T10:18:47.862 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T10:18:47.863 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T10:18:47.864 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T10:18:47.865 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T10:18:47.866 INFO:teuthology.orchestra.run.smithi190.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T10:18:47.867 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T10:18:47.867 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T10:18:47.867 INFO:teuthology.orchestra.run.smithi190.stdout:Installing weak dependencies: 2024-04-21T10:18:47.867 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T10:18:47.867 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:18:47.867 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-04-21T10:18:47.867 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-04-21T10:18:47.867 INFO:teuthology.orchestra.run.smithi190.stdout:Install 148 Packages 2024-04-21T10:18:47.867 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 2 Packages 2024-04-21T10:18:47.867 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:18:47.868 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 218 M 2024-04-21T10:18:47.868 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-04-21T10:18:47.952 INFO:teuthology.orchestra.run.smithi026.stdout:(122/150): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-21T10:18:47.976 INFO:teuthology.orchestra.run.smithi169.stdout:(43/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 232 kB/s | 253 kB 00:01 2024-04-21T10:18:47.977 INFO:teuthology.orchestra.run.smithi026.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-21T10:18:48.036 INFO:teuthology.orchestra.run.smithi026.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-21T10:18:48.064 INFO:teuthology.orchestra.run.smithi026.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 206 kB/s | 18 kB 00:00 2024-04-21T10:18:48.114 INFO:teuthology.orchestra.run.smithi026.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 299 kB/s | 23 kB 00:00 2024-04-21T10:18:48.139 INFO:teuthology.orchestra.run.smithi026.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-21T10:18:48.189 INFO:teuthology.orchestra.run.smithi026.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-04-21T10:18:48.248 INFO:teuthology.orchestra.run.smithi026.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-21T10:18:48.265 INFO:teuthology.orchestra.run.smithi026.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 614 kB/s | 46 kB 00:00 2024-04-21T10:18:48.332 INFO:teuthology.orchestra.run.smithi026.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 944 kB/s | 79 kB 00:00 2024-04-21T10:18:48.349 INFO:teuthology.orchestra.run.smithi026.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-04-21T10:18:48.424 INFO:teuthology.orchestra.run.smithi026.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-21T10:18:48.450 INFO:teuthology.orchestra.run.smithi026.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-21T10:18:48.516 INFO:teuthology.orchestra.run.smithi026.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 982 kB/s | 90 kB 00:00 2024-04-21T10:18:48.542 INFO:teuthology.orchestra.run.smithi026.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-04-21T10:18:48.609 INFO:teuthology.orchestra.run.smithi026.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-04-21T10:18:48.634 INFO:teuthology.orchestra.run.smithi026.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-21T10:18:48.676 INFO:teuthology.orchestra.run.smithi169.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 119 kB/s | 106 kB 00:00 2024-04-21T10:18:48.692 INFO:teuthology.orchestra.run.smithi026.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 712 kB/s | 59 kB 00:00 2024-04-21T10:18:48.718 INFO:teuthology.orchestra.run.smithi026.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2024-04-21T10:18:48.768 INFO:teuthology.orchestra.run.smithi026.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-04-21T10:18:48.810 INFO:teuthology.orchestra.run.smithi026.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-21T10:18:48.869 INFO:teuthology.orchestra.run.smithi026.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 896 kB/s | 90 kB 00:00 2024-04-21T10:18:48.919 INFO:teuthology.orchestra.run.smithi026.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-21T10:18:48.943 INFO:teuthology.orchestra.run.smithi169.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 140 kB/s | 135 kB 00:00 2024-04-21T10:18:48.969 INFO:teuthology.orchestra.run.smithi026.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-21T10:18:49.020 INFO:teuthology.orchestra.run.smithi026.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-21T10:18:49.077 INFO:teuthology.orchestra.run.smithi190.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-21T10:18:49.113 INFO:teuthology.orchestra.run.smithi026.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-21T10:18:49.611 INFO:teuthology.orchestra.run.smithi190.stdout:(2/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 6.3 MB/s | 5.4 MB 00:00 2024-04-21T10:18:49.669 INFO:teuthology.orchestra.run.smithi190.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 2.4 MB/s | 145 kB 00:00 2024-04-21T10:18:49.686 INFO:teuthology.orchestra.run.smithi026.stdout:(148/150): librbd1-19.0.0-3154.ga630765c.el9.x8 5.5 MB/s | 3.1 MB 00:00 2024-04-21T10:18:49.710 INFO:teuthology.orchestra.run.smithi169.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 121 kB/s | 125 kB 00:01 2024-04-21T10:18:49.745 INFO:teuthology.orchestra.run.smithi026.stdout:(149/150): librados2-19.0.0-3154.ga630765c.el9. 4.7 MB/s | 3.4 MB 00:00 2024-04-21T10:18:49.911 INFO:teuthology.orchestra.run.smithi190.stdout:(4/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 9.8 MB/s | 2.4 MB 00:00 2024-04-21T10:18:49.960 INFO:teuthology.orchestra.run.smithi169.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2024-04-21T10:18:50.003 INFO:teuthology.orchestra.run.smithi190.stdout:(5/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 962 kB/s | 890 kB 00:00 2024-04-21T10:18:50.052 INFO:teuthology.orchestra.run.smithi169.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-21T10:18:50.077 INFO:teuthology.orchestra.run.smithi169.stdout:(49/150): python3-urllib3-1.26.5-5.el9.noarch.r 189 kB/s | 215 kB 00:01 2024-04-21T10:18:50.086 INFO:teuthology.orchestra.run.smithi190.stdout:(6/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 8.5 MB/s | 1.5 MB 00:00 2024-04-21T10:18:50.119 INFO:teuthology.orchestra.run.smithi169.stdout:(50/150): python3-cryptography-36.0.1-4.el9.x86 473 kB/s | 1.2 MB 00:02 2024-04-21T10:18:50.146 INFO:teuthology.orchestra.run.smithi169.stdout:(51/150): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-21T10:18:50.220 INFO:teuthology.orchestra.run.smithi169.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-04-21T10:18:50.270 INFO:teuthology.orchestra.run.smithi169.stdout:(53/150): elfutils-libelf-devel-0.190-2.el9.x86 112 kB/s | 21 kB 00:00 2024-04-21T10:18:50.328 INFO:teuthology.orchestra.run.smithi169.stdout:(54/150): flexiblas-3.0.4-8.el9.x86_64.rpm 209 kB/s | 33 kB 00:00 2024-04-21T10:18:50.345 INFO:teuthology.orchestra.run.smithi169.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 245 kB/s | 18 kB 00:00 2024-04-21T10:18:50.412 INFO:teuthology.orchestra.run.smithi169.stdout:(56/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-04-21T10:18:50.429 INFO:teuthology.orchestra.run.smithi169.stdout:(57/150): libaio-devel-0.3.111-13.el9.x86_64.rp 110 kB/s | 11 kB 00:00 2024-04-21T10:18:50.463 INFO:teuthology.orchestra.run.smithi169.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-21T10:18:50.479 INFO:teuthology.orchestra.run.smithi026.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.4 MB/s | 16 MB 00:04 2024-04-21T10:18:50.485 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-04-21T10:18:50.485 INFO:teuthology.orchestra.run.smithi026.stdout:Total 13 MB/s | 218 MB 00:17 2024-04-21T10:18:50.521 INFO:teuthology.orchestra.run.smithi169.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-21T10:18:50.547 INFO:teuthology.orchestra.run.smithi169.stdout:(60/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 797 kB/s | 160 kB 00:00 2024-04-21T10:18:50.580 INFO:teuthology.orchestra.run.smithi169.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 433 kB/s | 25 kB 00:00 2024-04-21T10:18:50.597 INFO:teuthology.orchestra.run.smithi169.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-04-21T10:18:50.631 INFO:teuthology.orchestra.run.smithi169.stdout:(63/150): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-21T10:18:50.656 INFO:teuthology.orchestra.run.smithi169.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-21T10:18:50.681 INFO:teuthology.orchestra.run.smithi169.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-21T10:18:50.707 INFO:teuthology.orchestra.run.smithi169.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 466 kB/s | 35 kB 00:00 2024-04-21T10:18:50.787 INFO:teuthology.orchestra.run.smithi190.stdout:(7/150): ceph-common-19.0.0-3154.ga630765c.el9. 10 MB/s | 21 MB 00:02 2024-04-21T10:18:50.857 INFO:teuthology.orchestra.run.smithi169.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 24 MB/s | 4.1 MB 00:00 2024-04-21T10:18:50.899 INFO:teuthology.orchestra.run.smithi169.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-04-21T10:18:50.966 INFO:teuthology.orchestra.run.smithi169.stdout:(69/150): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-21T10:18:51.000 INFO:teuthology.orchestra.run.smithi169.stdout:(70/150): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-04-21T10:18:51.025 INFO:teuthology.orchestra.run.smithi169.stdout:(71/150): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-21T10:18:51.108 INFO:teuthology.orchestra.run.smithi169.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-04-21T10:18:51.134 INFO:teuthology.orchestra.run.smithi169.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 806 kB/s | 87 kB 00:00 2024-04-21T10:18:51.217 INFO:teuthology.orchestra.run.smithi169.stdout:(74/150): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-04-21T10:18:51.268 INFO:teuthology.orchestra.run.smithi169.stdout:(75/150): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-04-21T10:18:51.293 INFO:teuthology.orchestra.run.smithi169.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-21T10:18:51.377 INFO:teuthology.orchestra.run.smithi169.stdout:(77/150): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-04-21T10:18:51.402 INFO:teuthology.orchestra.run.smithi169.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-04-21T10:18:51.427 INFO:teuthology.orchestra.run.smithi169.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-04-21T10:18:51.452 INFO:teuthology.orchestra.run.smithi169.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-21T10:18:51.478 INFO:teuthology.orchestra.run.smithi169.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 460 kB/s | 35 kB 00:00 2024-04-21T10:18:51.504 INFO:teuthology.orchestra.run.smithi190.stdout:(8/150): ceph-osd-19.0.0-3154.ga630765c.el9.x86 12 MB/s | 17 MB 00:01 2024-04-21T10:18:51.545 INFO:teuthology.orchestra.run.smithi169.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-21T10:18:51.545 INFO:teuthology.orchestra.run.smithi190.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 609 kB/s | 25 kB 00:00 2024-04-21T10:18:51.570 INFO:teuthology.orchestra.run.smithi169.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-21T10:18:51.583 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-04-21T10:18:51.645 INFO:teuthology.orchestra.run.smithi169.stdout:(84/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-04-21T10:18:51.670 INFO:teuthology.orchestra.run.smithi169.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 615 kB/s | 77 kB 00:00 2024-04-21T10:18:51.671 INFO:teuthology.orchestra.run.smithi190.stdout:(10/150): ceph-radosgw-19.0.0-3154.ga630765c.el 12 MB/s | 11 MB 00:00 2024-04-21T10:18:51.715 INFO:teuthology.orchestra.run.smithi169.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 291 kB/s | 42 kB 00:00 2024-04-21T10:18:51.723 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-04-21T10:18:51.723 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-04-21T10:18:51.746 INFO:teuthology.orchestra.run.smithi169.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-21T10:18:51.746 INFO:teuthology.orchestra.run.smithi190.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 2.9 MB/s | 225 kB 00:00 2024-04-21T10:18:51.779 INFO:teuthology.orchestra.run.smithi169.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-21T10:18:51.805 INFO:teuthology.orchestra.run.smithi169.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 279 kB 00:00 2024-04-21T10:18:51.813 INFO:teuthology.orchestra.run.smithi190.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 405 kB/s | 27 kB 00:00 2024-04-21T10:18:51.830 INFO:teuthology.orchestra.run.smithi169.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 609 kB/s | 51 kB 00:00 2024-04-21T10:18:51.855 INFO:teuthology.orchestra.run.smithi169.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-04-21T10:18:51.914 INFO:teuthology.orchestra.run.smithi190.stdout:(13/150): ceph-mon-19.0.0-3154.ga630765c.el9.x8 2.5 MB/s | 4.7 MB 00:01 2024-04-21T10:18:51.922 INFO:teuthology.orchestra.run.smithi169.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-04-21T10:18:51.939 INFO:teuthology.orchestra.run.smithi190.stdout:(14/150): libcephfs-devel-19.0.0-3154.ga630765c 253 kB/s | 32 kB 00:00 2024-04-21T10:18:51.964 INFO:teuthology.orchestra.run.smithi169.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-04-21T10:18:52.014 INFO:teuthology.orchestra.run.smithi190.stdout:(15/150): libcephsqlite-19.0.0-3154.ga630765c.e 2.1 MB/s | 163 kB 00:00 2024-04-21T10:18:52.023 INFO:teuthology.orchestra.run.smithi169.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-04-21T10:18:52.056 INFO:teuthology.orchestra.run.smithi169.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 390 kB/s | 36 kB 00:00 2024-04-21T10:18:52.089 INFO:teuthology.orchestra.run.smithi190.stdout:(16/150): librados-devel-19.0.0-3154.ga630765c. 1.7 MB/s | 127 kB 00:00 2024-04-21T10:18:52.106 INFO:teuthology.orchestra.run.smithi169.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-04-21T10:18:52.132 INFO:teuthology.orchestra.run.smithi190.stdout:(17/150): libcephfs2-19.0.0-3154.ga630765c.el9. 3.4 MB/s | 746 kB 00:00 2024-04-21T10:18:52.174 INFO:teuthology.orchestra.run.smithi190.stdout:(18/150): libradosstriper1-19.0.0-3154.ga630765 5.8 MB/s | 501 kB 00:00 2024-04-21T10:18:52.249 INFO:teuthology.orchestra.run.smithi190.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 601 kB/s | 45 kB 00:00 2024-04-21T10:18:52.266 INFO:teuthology.orchestra.run.smithi169.stdout:(97/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-21T10:18:52.291 INFO:teuthology.orchestra.run.smithi169.stdout:(98/150): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 305 kB 00:00 2024-04-21T10:18:52.308 INFO:teuthology.orchestra.run.smithi169.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 288 kB/s | 58 kB 00:00 2024-04-21T10:18:52.324 INFO:teuthology.orchestra.run.smithi190.stdout:(20/150): python3-ceph-common-19.0.0-3154.ga630 1.8 MB/s | 135 kB 00:00 2024-04-21T10:18:52.333 INFO:teuthology.orchestra.run.smithi169.stdout:(100/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-04-21T10:18:52.350 INFO:teuthology.orchestra.run.smithi169.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-21T10:18:52.367 INFO:teuthology.orchestra.run.smithi169.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 759 kB/s | 45 kB 00:00 2024-04-21T10:18:52.402 INFO:teuthology.orchestra.run.smithi190.stdout:(21/150): python3-cephfs-19.0.0-3154.ga630765c. 2.1 MB/s | 162 kB 00:00 2024-04-21T10:18:52.491 INFO:teuthology.orchestra.run.smithi190.stdout:(22/150): python3-rados-19.0.0-3154.ga630765c.e 3.4 MB/s | 321 kB 00:00 2024-04-21T10:18:52.575 INFO:teuthology.orchestra.run.smithi190.stdout:(23/150): python3-rbd-19.0.0-3154.ga630765c.el9 3.5 MB/s | 298 kB 00:00 2024-04-21T10:18:52.642 INFO:teuthology.orchestra.run.smithi169.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-04-21T10:18:52.650 INFO:teuthology.orchestra.run.smithi190.stdout:(24/150): python3-rgw-19.0.0-3154.ga630765c.el9 1.3 MB/s | 100 kB 00:00 2024-04-21T10:18:52.701 INFO:teuthology.orchestra.run.smithi169.stdout:(104/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-04-21T10:18:52.726 INFO:teuthology.orchestra.run.smithi190.stdout:(25/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-21T10:18:53.059 INFO:teuthology.orchestra.run.smithi190.stdout:(26/150): librgw2-19.0.0-3154.ga630765c.el9.x86 5.8 MB/s | 5.4 MB 00:00 2024-04-21T10:18:53.109 INFO:teuthology.orchestra.run.smithi169.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 48 kB/s | 19 kB 00:00 2024-04-21T10:18:53.110 INFO:teuthology.orchestra.run.smithi190.stdout:(27/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 8.1 MB/s | 3.1 MB 00:00 2024-04-21T10:18:53.135 INFO:teuthology.orchestra.run.smithi190.stdout:(28/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-21T10:18:53.160 INFO:teuthology.orchestra.run.smithi190.stdout:(29/150): ceph-grafana-dashboards-19.0.0-3154.g 680 kB/s | 34 kB 00:00 2024-04-21T10:18:53.210 INFO:teuthology.orchestra.run.smithi190.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 1.8 MB/s | 142 kB 00:00 2024-04-21T10:18:53.335 INFO:teuthology.orchestra.run.smithi169.stdout:(106/150): abseil-cpp-20211102.0-3.el9.x86_64.r 567 kB/s | 548 kB 00:00 2024-04-21T10:18:53.460 INFO:teuthology.orchestra.run.smithi169.stdout:(107/150): gperftools-libs-2.9.1-2.el9.x86_64.r 378 kB/s | 309 kB 00:00 2024-04-21T10:18:53.493 INFO:teuthology.orchestra.run.smithi169.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 164 kB/s | 26 kB 00:00 2024-04-21T10:18:53.552 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-04-21T10:18:53.552 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-04-21T10:18:53.586 INFO:teuthology.orchestra.run.smithi190.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 8.3 MB/s | 3.5 MB 00:00 2024-04-21T10:18:53.619 INFO:teuthology.orchestra.run.smithi169.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 306 kB/s | 49 kB 00:00 2024-04-21T10:18:53.652 INFO:teuthology.orchestra.run.smithi169.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 424 kB/s | 67 kB 00:00 2024-04-21T10:18:53.653 INFO:teuthology.orchestra.run.smithi190.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 3.7 MB/s | 249 kB 00:00 2024-04-21T10:18:53.711 INFO:teuthology.orchestra.run.smithi190.stdout:(33/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 846 kB/s | 49 kB 00:00 2024-04-21T10:18:53.778 INFO:teuthology.orchestra.run.smithi169.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 953 kB/s | 151 kB 00:00 2024-04-21T10:18:53.778 INFO:teuthology.orchestra.run.smithi190.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3154.ga 247 kB/s | 16 kB 00:00 2024-04-21T10:18:53.862 INFO:teuthology.orchestra.run.smithi190.stdout:(35/150): ceph-volume-19.0.0-3154.ga630765c.el9 3.2 MB/s | 273 kB 00:00 2024-04-21T10:18:53.953 INFO:teuthology.orchestra.run.smithi169.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-04-21T10:18:54.004 INFO:teuthology.orchestra.run.smithi190.stdout:(36/150): cephadm-19.0.0-3154.ga630765c.el9.noa 5.3 MB/s | 762 kB 00:00 2024-04-21T10:18:54.070 INFO:teuthology.orchestra.run.smithi169.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-04-21T10:18:54.070 INFO:teuthology.orchestra.run.smithi190.stdout:(37/150): cephfs-top-19.0.0-3154.ga630765c.el9. 274 kB/s | 18 kB 00:00 2024-04-21T10:18:54.112 INFO:teuthology.orchestra.run.smithi169.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 274 kB/s | 43 kB 00:00 2024-04-21T10:18:54.263 INFO:teuthology.orchestra.run.smithi169.stdout:(115/150): libarrow-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-04-21T10:18:54.287 INFO:teuthology.orchestra.run.smithi169.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 85 kB/s | 15 kB 00:00 2024-04-21T10:18:54.296 INFO:teuthology.orchestra.run.smithi190.stdout:(38/150): ceph-mgr-diskprediction-local-19.0.0- 6.8 MB/s | 7.4 MB 00:01 2024-04-21T10:18:54.313 INFO:teuthology.orchestra.run.smithi169.stdout:(117/150): python3-cachetools-4.2.4-1.el9.noarc 133 kB/s | 32 kB 00:00 2024-04-21T10:18:54.321 INFO:teuthology.orchestra.run.smithi190.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 321 kB/s | 80 kB 00:00 2024-04-21T10:18:54.421 INFO:teuthology.orchestra.run.smithi169.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-04-21T10:18:54.446 INFO:teuthology.orchestra.run.smithi169.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-04-21T10:18:54.480 INFO:teuthology.orchestra.run.smithi169.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-04-21T10:18:54.580 INFO:teuthology.orchestra.run.smithi169.stdout:(121/150): python3-colorama-0.4.4-12.el9.noarch 222 kB/s | 35 kB 00:00 2024-04-21T10:18:54.605 INFO:teuthology.orchestra.run.smithi169.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 1.2 MB/s | 203 kB 00:00 2024-04-21T10:18:54.739 INFO:teuthology.orchestra.run.smithi169.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 910 kB/s | 144 kB 00:00 2024-04-21T10:18:54.746 INFO:teuthology.orchestra.run.smithi190.stdout:(40/150): ceph-test-19.0.0-3154.ga630765c.el9.x 15 MB/s | 49 MB 00:03 2024-04-21T10:18:54.764 INFO:teuthology.orchestra.run.smithi169.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 67 kB/s | 11 kB 00:00 2024-04-21T10:18:54.806 INFO:teuthology.orchestra.run.smithi169.stdout:(125/150): python3-grpcio-1.46.7-10.el9.x86_64. 6.3 MB/s | 2.0 MB 00:00 2024-04-21T10:18:54.840 INFO:teuthology.orchestra.run.smithi190.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-04-21T10:18:54.907 INFO:teuthology.orchestra.run.smithi169.stdout:(126/150): python3-jaraco-classes-3.2.1-5.el9.n 106 kB/s | 18 kB 00:00 2024-04-21T10:18:54.932 INFO:teuthology.orchestra.run.smithi169.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 138 kB/s | 23 kB 00:00 2024-04-21T10:18:54.957 INFO:teuthology.orchestra.run.smithi169.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 129 kB/s | 19 kB 00:00 2024-04-21T10:18:54.966 INFO:teuthology.orchestra.run.smithi190.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-21T10:18:54.991 INFO:teuthology.orchestra.run.smithi190.stdout:(43/150): libquadmath-11.4.1-3.el9.x86_64.rpm 292 kB/s | 195 kB 00:00 2024-04-21T10:18:55.033 INFO:teuthology.orchestra.run.smithi190.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-21T10:18:55.058 INFO:teuthology.orchestra.run.smithi190.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-21T10:18:55.066 INFO:teuthology.orchestra.run.smithi169.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 124 kB/s | 20 kB 00:00 2024-04-21T10:18:55.083 INFO:teuthology.orchestra.run.smithi190.stdout:(46/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 753 kB/s | 253 kB 00:00 2024-04-21T10:18:55.108 INFO:teuthology.orchestra.run.smithi169.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-04-21T10:18:55.108 INFO:teuthology.orchestra.run.smithi190.stdout:(47/150): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-21T10:18:55.133 INFO:teuthology.orchestra.run.smithi169.stdout:(131/150): python3-logutils-0.3.5-21.el9.noarch 264 kB/s | 46 kB 00:00 2024-04-21T10:18:55.142 INFO:teuthology.orchestra.run.smithi190.stdout:(48/150): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-21T10:18:55.225 INFO:teuthology.orchestra.run.smithi169.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 496 kB/s | 79 kB 00:00 2024-04-21T10:18:55.267 INFO:teuthology.orchestra.run.smithi169.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 364 kB/s | 58 kB 00:00 2024-04-21T10:18:55.292 INFO:teuthology.orchestra.run.smithi169.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-04-21T10:18:55.317 INFO:teuthology.orchestra.run.smithi190.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 611 kB/s | 107 kB 00:00 2024-04-21T10:18:55.368 INFO:teuthology.orchestra.run.smithi190.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 432 kB/s | 21 kB 00:00 2024-04-21T10:18:55.384 INFO:teuthology.orchestra.run.smithi169.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 103 kB/s | 16 kB 00:00 2024-04-21T10:18:55.401 INFO:teuthology.orchestra.run.smithi190.stdout:(51/150): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-04-21T10:18:55.426 INFO:teuthology.orchestra.run.smithi169.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 566 kB/s | 90 kB 00:00 2024-04-21T10:18:55.435 INFO:teuthology.orchestra.run.smithi190.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-04-21T10:18:55.451 INFO:teuthology.orchestra.run.smithi169.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 177 kB/s | 28 kB 00:00 2024-04-21T10:18:55.460 INFO:teuthology.orchestra.run.smithi190.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-04-21T10:18:55.485 INFO:teuthology.orchestra.run.smithi190.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-21T10:18:55.519 INFO:teuthology.orchestra.run.smithi190.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-04-21T10:18:55.543 INFO:teuthology.orchestra.run.smithi169.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 194 kB/s | 31 kB 00:00 2024-04-21T10:18:55.544 INFO:teuthology.orchestra.run.smithi190.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-04-21T10:18:55.578 INFO:teuthology.orchestra.run.smithi190.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-21T10:18:55.585 INFO:teuthology.orchestra.run.smithi169.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-21T10:18:55.603 INFO:teuthology.orchestra.run.smithi190.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-04-21T10:18:55.610 INFO:teuthology.orchestra.run.smithi169.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 373 kB/s | 59 kB 00:00 2024-04-21T10:18:55.702 INFO:teuthology.orchestra.run.smithi169.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 225 kB/s | 36 kB 00:00 2024-04-21T10:18:55.711 INFO:teuthology.orchestra.run.smithi190.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-04-21T10:18:55.737 INFO:teuthology.orchestra.run.smithi190.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-04-21T10:18:55.744 INFO:teuthology.orchestra.run.smithi169.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 321 kB/s | 51 kB 00:00 2024-04-21T10:18:55.762 INFO:teuthology.orchestra.run.smithi190.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 506 kB/s | 25 kB 00:00 2024-04-21T10:18:55.769 INFO:teuthology.orchestra.run.smithi169.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-04-21T10:18:55.787 INFO:teuthology.orchestra.run.smithi190.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-21T10:18:55.838 INFO:teuthology.orchestra.run.smithi190.stdout:(63/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-04-21T10:18:55.862 INFO:teuthology.orchestra.run.smithi169.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 564 kB/s | 90 kB 00:00 2024-04-21T10:18:55.863 INFO:teuthology.orchestra.run.smithi190.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-21T10:18:55.888 INFO:teuthology.orchestra.run.smithi190.stdout:(65/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-21T10:18:55.903 INFO:teuthology.orchestra.run.smithi169.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-04-21T10:18:55.905 INFO:teuthology.orchestra.run.smithi190.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 525 kB/s | 35 kB 00:00 2024-04-21T10:18:55.929 INFO:teuthology.orchestra.run.smithi169.stdout:(146/150): python3-zc-lockfile-2.0-10.el9.noarc 126 kB/s | 20 kB 00:00 2024-04-21T10:18:56.021 INFO:teuthology.orchestra.run.smithi169.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-21T10:18:56.072 INFO:teuthology.orchestra.run.smithi190.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-21T10:18:56.189 INFO:teuthology.orchestra.run.smithi190.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-04-21T10:18:56.274 INFO:teuthology.orchestra.run.smithi190.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 1.5 MB/s | 133 kB 00:00 2024-04-21T10:18:56.305 INFO:teuthology.orchestra.run.smithi169.stdout:(148/150): librbd1-19.0.0-3154.ga630765c.el9.x8 11 MB/s | 3.1 MB 00:00 2024-04-21T10:18:56.389 INFO:teuthology.orchestra.run.smithi190.stdout:(70/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 8.3 MB/s | 4.1 MB 00:00 2024-04-21T10:18:56.456 INFO:teuthology.orchestra.run.smithi190.stdout:(71/150): openblas-openmp-0.3.21-2.el9.x86_64.r 8.0 MB/s | 4.7 MB 00:00 2024-04-21T10:18:56.498 INFO:teuthology.orchestra.run.smithi190.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-04-21T10:18:56.524 INFO:teuthology.orchestra.run.smithi190.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-21T10:18:56.582 INFO:teuthology.orchestra.run.smithi190.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-21T10:18:56.649 INFO:teuthology.orchestra.run.smithi190.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-21T10:18:56.716 INFO:teuthology.orchestra.run.smithi190.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 385 kB/s | 26 kB 00:00 2024-04-21T10:18:56.779 INFO:teuthology.orchestra.run.smithi169.stdout:(149/150): librados2-19.0.0-3154.ga630765c.el9. 4.0 MB/s | 3.4 MB 00:00 2024-04-21T10:18:56.816 INFO:teuthology.orchestra.run.smithi190.stdout:(77/150): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-21T10:18:56.842 INFO:teuthology.orchestra.run.smithi190.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-04-21T10:18:56.875 INFO:teuthology.orchestra.run.smithi190.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-21T10:18:56.901 INFO:teuthology.orchestra.run.smithi190.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-04-21T10:18:56.976 INFO:teuthology.orchestra.run.smithi190.stdout:(81/150): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-04-21T10:18:57.001 INFO:teuthology.orchestra.run.smithi190.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-21T10:18:57.060 INFO:teuthology.orchestra.run.smithi190.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-21T10:18:57.085 INFO:teuthology.orchestra.run.smithi190.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 924 kB/s | 77 kB 00:00 2024-04-21T10:18:57.119 INFO:teuthology.orchestra.run.smithi190.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-04-21T10:18:57.153 INFO:teuthology.orchestra.run.smithi190.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-04-21T10:18:57.186 INFO:teuthology.orchestra.run.smithi190.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-21T10:18:57.228 INFO:teuthology.orchestra.run.smithi190.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-21T10:18:57.253 INFO:teuthology.orchestra.run.smithi190.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 764 kB/s | 51 kB 00:00 2024-04-21T10:18:57.320 INFO:teuthology.orchestra.run.smithi190.stdout:(90/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-04-21T10:18:57.345 INFO:teuthology.orchestra.run.smithi190.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 458 kB/s | 54 kB 00:00 2024-04-21T10:18:57.378 INFO:teuthology.orchestra.run.smithi190.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-04-21T10:18:57.404 INFO:teuthology.orchestra.run.smithi190.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 754 kB/s | 44 kB 00:00 2024-04-21T10:18:57.437 INFO:teuthology.orchestra.run.smithi190.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-21T10:18:57.463 INFO:teuthology.orchestra.run.smithi190.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-04-21T10:18:57.496 INFO:teuthology.orchestra.run.smithi190.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-04-21T10:18:57.530 INFO:teuthology.orchestra.run.smithi190.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-21T10:18:57.555 INFO:teuthology.orchestra.run.smithi190.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 985 kB/s | 58 kB 00:00 2024-04-21T10:18:57.589 INFO:teuthology.orchestra.run.smithi190.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-21T10:18:57.614 INFO:teuthology.orchestra.run.smithi190.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-21T10:18:57.640 INFO:teuthology.orchestra.run.smithi190.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-04-21T10:18:57.998 INFO:teuthology.orchestra.run.smithi190.stdout:(102/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-21T10:18:58.382 INFO:teuthology.orchestra.run.smithi190.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-04-21T10:18:58.640 INFO:teuthology.orchestra.run.smithi190.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 928 kB/s | 548 kB 00:00 2024-04-21T10:18:58.899 INFO:teuthology.orchestra.run.smithi190.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-21T10:18:58.966 INFO:teuthology.orchestra.run.smithi190.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 529 kB/s | 309 kB 00:00 2024-04-21T10:18:59.224 INFO:teuthology.orchestra.run.smithi190.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 101 kB/s | 26 kB 00:00 2024-04-21T10:18:59.545 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-04-21T10:18:59.566 INFO:teuthology.orchestra.run.smithi190.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-21T10:18:59.592 INFO:teuthology.orchestra.run.smithi190.stdout:(109/150): protobuf-compiler-3.14.0-13.el9.x86_ 442 kB/s | 863 kB 00:01 2024-04-21T10:18:59.613 INFO:teuthology.orchestra.run.smithi169.stdout:(150/150): thrift-0.15.0-2.el9.x86_64.rpm 437 kB/s | 1.6 MB 00:03 2024-04-21T10:18:59.619 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-04-21T10:18:59.620 INFO:teuthology.orchestra.run.smithi169.stdout:Total 8.7 MB/s | 218 MB 00:24 2024-04-21T10:18:59.668 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T10:18:59.840 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T10:18:59.885 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T10:18:59.917 INFO:teuthology.orchestra.run.smithi190.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 192 kB/s | 67 kB 00:00 2024-04-21T10:19:00.034 INFO:teuthology.orchestra.run.smithi190.stdout:(111/150): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-04-21T10:19:00.059 INFO:teuthology.orchestra.run.smithi190.stdout:(112/150): luarocks-3.9.2-1.el9.noarch.rpm 324 kB/s | 151 kB 00:00 2024-04-21T10:19:00.238 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T10:19:00.340 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T10:19:00.401 INFO:teuthology.orchestra.run.smithi190.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 127 kB/s | 43 kB 00:00 2024-04-21T10:19:00.448 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T10:19:00.475 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T10:19:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T10:19:00.568 INFO:teuthology.orchestra.run.smithi190.stdout:(114/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.3 MB/s | 838 kB 00:00 2024-04-21T10:19:00.609 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T10:19:00.626 INFO:teuthology.orchestra.run.smithi190.stdout:(115/150): python3-asyncssh-2.13.2-5.el9.noarch 925 kB/s | 548 kB 00:00 2024-04-21T10:19:00.637 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T10:19:00.663 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T10:19:00.668 INFO:teuthology.orchestra.run.smithi190.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 121 kB/s | 32 kB 00:00 2024-04-21T10:19:00.687 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-04-21T10:19:00.813 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-04-21T10:19:00.813 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-04-21T10:19:00.824 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T10:19:00.852 INFO:teuthology.orchestra.run.smithi190.stdout:(117/150): python3-certifi-2021.10.8-2.el9.noar 53 kB/s | 15 kB 00:00 2024-04-21T10:19:00.878 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T10:19:00.905 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T10:19:01.010 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T10:19:01.052 INFO:teuthology.orchestra.run.smithi190.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 404 kB/s | 172 kB 00:00 2024-04-21T10:19:01.056 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T10:19:01.119 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T10:19:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T10:19:01.167 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T10:19:01.169 INFO:teuthology.orchestra.run.smithi190.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 716 kB/s | 358 kB 00:00 2024-04-21T10:19:01.319 INFO:teuthology.orchestra.run.smithi190.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-21T10:19:01.358 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T10:19:01.361 INFO:teuthology.orchestra.run.smithi190.stdout:(121/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 595 kB/s | 303 kB 00:00 2024-04-21T10:19:01.390 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T10:19:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T10:19:01.600 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T10:19:01.631 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T10:19:01.667 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T10:19:01.678 INFO:teuthology.orchestra.run.smithi190.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 398 kB/s | 203 kB 00:00 2024-04-21T10:19:01.744 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T10:19:01.787 INFO:teuthology.orchestra.run.smithi190.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-04-21T10:19:01.791 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T10:19:01.838 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T10:19:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T10:19:01.878 INFO:teuthology.orchestra.run.smithi190.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 53 kB/s | 11 kB 00:00 2024-04-21T10:19:01.894 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T10:19:01.968 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T10:19:02.028 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T10:19:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T10:19:02.062 INFO:teuthology.orchestra.run.smithi190.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 64 kB/s | 18 kB 00:00 2024-04-21T10:19:02.099 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T10:19:02.104 INFO:teuthology.orchestra.run.smithi190.stdout:(126/150): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-04-21T10:19:02.146 INFO:teuthology.orchestra.run.smithi190.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 87 kB/s | 23 kB 00:00 2024-04-21T10:19:02.179 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T10:19:02.218 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T10:19:02.262 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T10:19:02.290 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T10:19:02.330 INFO:teuthology.orchestra.run.smithi190.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-04-21T10:19:02.367 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T10:19:02.371 INFO:teuthology.orchestra.run.smithi190.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-04-21T10:19:02.414 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T10:19:02.493 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T10:19:02.534 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-04-21T10:19:02.535 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-04-21T10:19:02.639 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T10:19:02.688 INFO:teuthology.orchestra.run.smithi190.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 129 kB/s | 46 kB 00:00 2024-04-21T10:19:02.693 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T10:19:02.705 INFO:teuthology.orchestra.run.smithi190.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 236 kB/s | 79 kB 00:00 2024-04-21T10:19:02.832 INFO:teuthology.orchestra.run.smithi190.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-04-21T10:19:03.031 INFO:teuthology.orchestra.run.smithi190.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 169 kB/s | 58 kB 00:00 2024-04-21T10:19:03.098 INFO:teuthology.orchestra.run.smithi190.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-04-21T10:19:03.207 INFO:teuthology.orchestra.run.smithi190.stdout:(135/150): python3-pecan-1.4.2-3.el9.noarch.rpm 544 kB/s | 272 kB 00:00 2024-04-21T10:19:03.266 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T10:19:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T10:19:03.365 INFO:teuthology.orchestra.run.smithi190.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 106 kB/s | 28 kB 00:00 2024-04-21T10:19:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T10:19:03.425 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T10:19:03.457 INFO:teuthology.orchestra.run.smithi190.stdout:(137/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 211 kB/s | 90 kB 00:00 2024-04-21T10:19:03.457 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T10:19:03.482 INFO:teuthology.orchestra.run.smithi190.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 112 kB/s | 31 kB 00:00 2024-04-21T10:19:03.644 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T10:19:03.729 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T10:19:03.757 INFO:teuthology.orchestra.run.smithi190.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 130 kB/s | 36 kB 00:00 2024-04-21T10:19:03.799 INFO:teuthology.orchestra.run.smithi190.stdout:(140/150): python3-routes-2.5.1-5.el9.noarch.rp 434 kB/s | 188 kB 00:00 2024-04-21T10:19:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T10:19:03.825 INFO:teuthology.orchestra.run.smithi190.stdout:(141/150): python3-rsa-4.9-2.el9.noarch.rpm 161 kB/s | 59 kB 00:00 2024-04-21T10:19:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T10:19:03.875 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T10:19:03.923 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T10:19:03.958 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T10:19:04.001 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T10:19:04.044 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T10:19:04.108 INFO:teuthology.orchestra.run.smithi190.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 145 kB/s | 51 kB 00:00 2024-04-21T10:19:04.167 INFO:teuthology.orchestra.run.smithi190.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 262 kB/s | 90 kB 00:00 2024-04-21T10:19:04.190 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T10:19:04.241 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T10:19:04.271 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T10:19:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T10:19:04.317 INFO:teuthology.orchestra.run.smithi190.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 444 kB/s | 230 kB 00:00 2024-04-21T10:19:04.412 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T10:19:04.434 INFO:teuthology.orchestra.run.smithi190.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-04-21T10:19:04.445 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T10:19:04.485 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T10:19:04.511 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T10:19:04.535 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T10:19:04.615 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T10:19:04.692 INFO:teuthology.orchestra.run.smithi190.stdout:(146/150): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-04-21T10:19:04.744 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T10:19:04.784 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T10:19:04.831 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T10:19:04.834 INFO:teuthology.orchestra.run.smithi190.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 371 kB/s | 192 kB 00:00 2024-04-21T10:19:04.864 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T10:19:04.909 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T10:19:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T10:19:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T10:19:05.107 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T10:19:05.142 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T10:19:05.151 INFO:teuthology.orchestra.run.smithi190.stdout:(148/150): librbd1-19.0.0-3154.ga630765c.el9.x8 9.9 MB/s | 3.1 MB 00:00 2024-04-21T10:19:05.175 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T10:19:05.185 INFO:teuthology.orchestra.run.smithi190.stdout:(149/150): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-04-21T10:19:05.218 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T10:19:05.251 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T10:19:05.285 INFO:teuthology.orchestra.run.smithi190.stdout:(150/150): librados2-19.0.0-3154.ga630765c.el9. 5.7 MB/s | 3.4 MB 00:00 2024-04-21T10:19:05.288 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-04-21T10:19:05.288 INFO:teuthology.orchestra.run.smithi190.stdout:Total 12 MB/s | 218 MB 00:17 2024-04-21T10:19:05.344 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T10:19:05.399 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T10:19:05.429 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T10:19:05.478 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T10:19:05.589 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T10:19:05.640 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T10:19:06.313 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-04-21T10:19:06.423 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T10:19:06.440 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-04-21T10:19:06.441 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-04-21T10:19:06.469 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T10:19:06.503 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T10:19:06.607 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T10:19:06.670 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T10:19:06.760 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T10:19:06.804 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T10:19:06.987 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T10:19:07.026 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T10:19:07.107 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T10:19:07.796 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T10:19:07.921 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T10:19:08.165 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-04-21T10:19:08.165 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-04-21T10:19:08.607 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-04-21T10:19:08.682 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T10:19:08.707 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T10:19:08.750 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T10:19:09.090 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T10:19:09.200 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T10:19:09.243 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T10:19:09.314 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T10:19:09.330 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T10:19:09.342 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T10:19:09.359 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T10:19:09.392 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T10:19:09.426 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T10:19:09.468 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T10:19:09.496 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T10:19:09.522 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T10:19:09.679 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T10:19:09.720 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T10:19:09.728 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T10:19:09.756 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T10:19:09.762 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T10:19:09.843 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T10:19:09.854 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T10:19:09.865 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T10:19:09.884 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T10:19:09.911 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T10:19:09.920 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T10:19:09.941 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T10:19:09.964 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T10:19:10.158 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T10:19:10.191 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T10:19:10.300 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T10:19:10.402 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T10:19:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T10:19:10.442 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T10:19:10.449 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T10:19:10.475 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T10:19:10.503 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T10:19:10.533 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T10:19:10.550 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T10:19:10.598 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T10:19:10.655 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T10:19:10.693 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T10:19:10.720 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T10:19:10.784 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T10:19:10.844 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T10:19:10.873 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T10:19:10.917 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T10:19:10.995 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T10:19:11.027 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T10:19:11.070 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T10:19:11.099 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T10:19:11.172 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T10:19:11.223 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T10:19:11.300 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T10:19:11.446 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T10:19:11.501 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T10:19:12.045 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T10:19:12.138 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T10:19:12.174 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T10:19:12.217 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T10:19:12.255 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T10:19:12.290 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T10:19:12.344 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T10:19:12.417 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T10:19:12.455 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T10:19:12.492 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T10:19:12.540 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T10:19:12.575 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T10:19:12.618 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T10:19:12.660 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T10:19:12.799 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T10:19:12.849 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T10:19:12.878 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T10:19:12.920 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T10:19:12.922 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T10:19:12.956 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T10:19:13.017 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T10:19:13.053 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T10:19:13.094 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T10:19:13.119 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T10:19:13.143 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T10:19:13.219 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T10:19:13.348 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T10:19:13.384 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T10:19:13.412 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T10:19:13.455 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T10:19:13.499 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T10:19:13.525 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T10:19:13.537 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T10:19:13.567 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T10:19:13.576 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T10:19:13.606 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T10:19:13.640 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T10:19:13.659 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T10:19:13.659 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T10:19:13.659 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T10:19:13.660 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:13.674 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T10:19:13.693 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T10:19:13.716 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T10:19:13.750 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T10:19:13.839 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T10:19:13.889 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T10:19:13.920 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T10:19:13.967 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T10:19:14.076 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T10:19:14.133 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T10:19:14.147 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-04-21T10:19:14.221 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T10:19:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T10:19:14.289 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T10:19:14.620 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T10:19:14.751 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T10:19:14.852 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T10:19:14.880 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T10:19:14.881 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T10:19:14.927 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T10:19:14.956 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T10:19:14.977 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T10:19:14.998 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T10:19:15.035 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T10:19:15.061 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T10:19:15.097 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T10:19:15.152 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T10:19:15.211 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T10:19:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T10:19:15.267 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T10:19:15.294 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T10:19:15.304 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T10:19:15.398 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T10:19:15.420 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T10:19:15.483 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T10:19:15.484 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T10:19:15.505 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T10:19:15.518 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T10:19:15.528 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T10:19:15.597 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T10:19:15.729 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T10:19:15.771 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T10:19:15.894 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T10:19:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T10:19:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T10:19:16.072 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T10:19:16.145 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T10:19:16.187 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T10:19:16.230 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T10:19:16.263 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T10:19:16.265 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T10:19:16.291 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T10:19:16.361 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T10:19:16.377 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T10:19:16.414 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T10:19:16.444 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T10:19:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T10:19:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T10:19:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T10:19:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T10:19:16.670 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T10:19:16.736 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T10:19:16.785 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T10:19:16.862 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T10:19:17.002 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T10:19:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T10:19:17.587 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T10:19:17.664 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T10:19:17.670 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T10:19:17.704 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T10:19:17.747 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T10:19:17.755 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T10:19:17.775 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T10:19:17.793 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T10:19:17.808 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T10:19:17.826 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T10:19:17.881 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T10:19:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T10:19:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T10:19:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T10:19:18.077 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T10:19:18.113 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T10:19:18.122 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T10:19:18.156 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T10:19:18.162 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T10:19:18.197 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T10:19:18.246 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T10:19:18.276 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T10:19:18.312 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T10:19:18.334 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T10:19:18.387 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T10:19:18.417 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T10:19:18.459 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T10:19:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T10:19:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T10:19:18.632 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T10:19:18.657 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T10:19:18.682 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T10:19:18.756 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T10:19:18.813 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T10:19:18.840 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T10:19:18.884 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T10:19:18.895 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T10:19:18.914 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T10:19:18.925 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T10:19:18.942 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T10:19:18.977 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T10:19:19.020 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T10:19:19.058 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T10:19:19.089 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T10:19:19.119 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T10:19:19.154 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T10:19:19.188 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T10:19:19.229 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T10:19:19.263 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T10:19:19.350 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T10:19:19.393 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T10:19:19.433 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T10:19:19.481 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T10:19:19.580 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T10:19:19.635 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T10:19:20.370 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T10:19:20.424 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T10:19:20.457 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T10:19:20.549 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T10:19:20.607 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T10:19:20.694 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T10:19:20.742 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T10:19:20.918 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T10:19:20.948 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T10:19:21.027 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T10:19:21.238 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T10:19:21.272 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T10:19:21.677 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T10:19:21.789 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T10:19:21.835 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T10:19:21.875 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T10:19:21.971 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T10:19:21.971 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T10:19:21.971 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T10:19:21.971 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:22.018 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T10:19:23.052 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T10:19:23.144 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T10:19:23.172 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T10:19:23.205 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T10:19:23.519 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T10:19:23.568 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T10:19:23.667 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T10:19:23.698 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T10:19:23.733 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T10:19:24.215 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T10:19:24.245 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T10:19:24.317 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T10:19:24.338 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T10:19:25.502 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T10:19:25.503 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-04-21T10:19:25.503 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-04-21T10:19:25.503 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-04-21T10:19:25.503 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-04-21T10:19:25.503 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-04-21T10:19:25.503 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /root 2024-04-21T10:19:25.503 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-04-21T10:19:25.503 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:25.688 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T10:19:25.704 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T10:19:25.705 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:25.705 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T10:19:25.705 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:26.626 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T10:19:26.668 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T10:19:27.209 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T10:19:27.232 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T10:19:27.255 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T10:19:27.259 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T10:19:27.259 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:27.259 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T10:19:27.259 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T10:19:27.259 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T10:19:27.260 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:27.337 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T10:19:27.337 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T10:19:27.337 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T10:19:27.337 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:19:27.364 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T10:19:27.491 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T10:19:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T10:19:27.515 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:27.516 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T10:19:27.516 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T10:19:27.516 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T10:19:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:27.975 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T10:19:27.999 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T10:19:27.999 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:27.999 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T10:19:27.999 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T10:19:27.999 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T10:19:27.999 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:28.106 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T10:19:28.164 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T10:19:28.228 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T10:19:28.421 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T10:19:28.485 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T10:19:28.565 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T10:19:28.662 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T10:19:28.804 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T10:19:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T10:19:28.894 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T10:19:28.919 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T10:19:29.374 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T10:19:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T10:19:29.426 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T10:19:29.481 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T10:19:29.507 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T10:19:30.600 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T10:19:30.634 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T10:19:31.647 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T10:19:31.676 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T10:19:31.786 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T10:19:31.955 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T10:19:31.986 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T10:19:32.010 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T10:19:32.011 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:32.011 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T10:19:32.011 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T10:19:32.011 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T10:19:32.011 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:32.062 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T10:19:32.074 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T10:19:32.175 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T10:19:32.236 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T10:19:33.211 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T10:19:33.239 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T10:19:33.264 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T10:19:33.264 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:33.265 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T10:19:33.265 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T10:19:33.265 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T10:19:33.265 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:33.307 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T10:19:33.330 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T10:19:33.331 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:33.331 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T10:19:33.331 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:33.383 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T10:19:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T10:19:33.407 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:33.407 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T10:19:33.407 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T10:19:33.407 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T10:19:33.407 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:33.523 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T10:19:33.523 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-04-21T10:19:33.524 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-04-21T10:19:33.524 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-04-21T10:19:33.524 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-04-21T10:19:33.524 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-04-21T10:19:33.524 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-04-21T10:19:33.524 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-04-21T10:19:33.524 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:33.714 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T10:19:33.724 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T10:19:33.730 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T10:19:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T10:19:33.730 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T10:19:33.749 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:33.749 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T10:19:33.749 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T10:19:33.749 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T10:19:33.749 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:35.217 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T10:19:35.243 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T10:19:35.243 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:35.243 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T10:19:35.243 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T10:19:35.244 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T10:19:35.244 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:35.468 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T10:19:35.490 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T10:19:35.490 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:35.490 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T10:19:35.491 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T10:19:35.491 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T10:19:35.491 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:35.944 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T10:19:35.966 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T10:19:35.967 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:35.967 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T10:19:35.967 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T10:19:35.967 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T10:19:35.967 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:36.073 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T10:19:36.122 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T10:19:36.177 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T10:19:36.362 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T10:19:36.411 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T10:19:36.488 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T10:19:36.577 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T10:19:36.710 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T10:19:36.753 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T10:19:36.785 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T10:19:36.812 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T10:19:37.224 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T10:19:37.245 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T10:19:37.277 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T10:19:37.338 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T10:19:37.368 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T10:19:38.412 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T10:19:38.444 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T10:19:38.566 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T10:19:38.566 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2024-04-21T10:19:38.566 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2024-04-21T10:19:38.566 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2024-04-21T10:19:38.566 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2024-04-21T10:19:38.567 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2024-04-21T10:19:38.567 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /root 2024-04-21T10:19:38.567 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2024-04-21T10:19:38.567 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:19:38.587 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T10:19:38.628 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T10:19:38.669 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T10:19:38.701 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T10:19:38.728 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T10:19:38.751 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T10:19:38.766 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T10:19:38.766 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:38.766 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T10:19:38.766 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:19:38.930 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T10:19:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T10:19:39.004 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T10:19:39.421 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T10:19:39.445 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T10:19:39.549 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T10:19:39.657 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T10:19:39.701 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T10:19:39.718 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T10:19:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T10:19:39.747 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T10:19:39.771 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T10:19:39.771 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:39.771 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T10:19:39.771 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T10:19:39.771 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T10:19:39.771 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:39.815 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T10:19:39.826 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T10:19:39.923 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T10:19:39.980 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T10:19:40.254 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T10:19:40.279 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T10:19:40.279 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:40.279 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T10:19:40.279 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T10:19:40.279 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T10:19:40.280 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:19:40.504 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T10:19:40.526 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T10:19:40.526 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:40.526 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T10:19:40.526 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T10:19:40.526 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T10:19:40.526 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:19:40.566 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T10:19:40.647 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T10:19:40.689 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T10:19:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T10:19:40.809 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T10:19:40.810 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T10:19:40.877 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T10:19:40.877 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T10:19:40.906 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T10:19:40.942 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T10:19:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T10:19:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T10:19:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T10:19:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T10:19:40.965 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T10:19:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T10:19:40.986 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:40.986 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T10:19:40.986 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T10:19:40.986 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T10:19:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:19:41.009 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T10:19:41.032 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T10:19:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:41.032 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T10:19:41.032 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:41.077 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T10:19:41.093 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T10:19:41.100 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T10:19:41.100 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:41.100 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T10:19:41.100 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T10:19:41.100 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T10:19:41.100 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:41.150 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T10:19:41.205 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T10:19:41.389 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T10:19:41.410 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T10:19:41.434 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T10:19:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T10:19:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T10:19:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T10:19:41.434 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:41.447 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T10:19:41.525 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T10:19:41.621 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T10:19:41.751 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T10:19:41.801 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T10:19:41.837 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T10:19:41.865 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T10:19:42.326 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T10:19:42.356 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T10:19:42.388 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T10:19:42.448 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T10:19:42.479 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T10:19:43.517 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T10:19:43.556 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T10:19:44.519 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T10:19:44.548 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T10:19:44.648 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T10:19:44.811 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T10:19:44.841 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T10:19:44.864 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T10:19:44.864 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:44.864 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T10:19:44.864 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T10:19:44.864 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T10:19:44.865 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:19:44.908 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T10:19:44.919 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T10:19:45.018 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T10:19:45.065 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T10:19:45.986 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T10:19:46.014 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T10:19:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T10:19:46.042 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T10:19:46.042 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:46.042 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T10:19:46.042 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T10:19:46.042 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T10:19:46.042 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:19:46.056 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T10:19:46.086 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T10:19:46.097 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T10:19:46.108 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T10:19:46.108 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:46.108 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T10:19:46.108 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:19:46.137 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T10:19:46.162 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T10:19:46.164 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T10:19:46.184 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T10:19:46.184 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:46.184 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T10:19:46.185 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T10:19:46.185 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T10:19:46.185 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:19:46.350 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T10:19:46.402 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T10:19:46.441 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T10:19:46.488 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T10:19:46.510 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T10:19:46.510 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T10:19:46.511 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T10:19:46.511 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T10:19:46.511 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T10:19:46.511 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:19:46.828 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T10:19:46.828 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T10:19:46.828 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T10:19:46.828 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T10:19:46.828 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T10:19:46.828 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T10:19:46.829 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T10:19:46.830 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T10:19:46.830 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T10:19:46.830 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T10:19:46.830 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T10:19:46.830 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T10:19:46.830 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T10:19:46.830 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T10:19:46.830 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T10:19:46.830 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T10:19:46.831 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T10:19:46.831 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T10:19:46.843 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T10:19:46.843 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T10:19:46.843 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T10:19:46.843 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T10:19:46.843 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T10:19:46.843 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T10:19:46.844 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T10:19:46.844 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T10:19:46.844 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T10:19:46.844 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T10:19:46.844 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T10:19:46.844 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T10:19:46.844 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T10:19:46.844 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T10:19:46.844 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T10:19:46.844 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T10:19:46.844 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T10:19:46.844 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T10:19:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T10:19:46.846 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T10:19:46.846 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T10:19:46.846 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T10:19:46.846 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T10:19:46.846 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T10:19:46.846 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T10:19:46.846 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T10:19:46.846 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T10:19:46.846 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T10:19:46.846 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T10:19:46.846 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T10:19:46.846 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T10:19:46.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T10:19:46.848 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T10:19:46.848 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T10:19:46.848 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T10:19:46.848 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T10:19:46.848 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T10:19:46.848 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T10:19:46.848 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T10:19:46.848 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T10:19:46.848 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T10:19:46.848 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T10:19:46.848 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T10:19:46.848 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T10:19:46.849 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T10:19:46.850 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T10:19:46.850 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T10:19:46.850 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T10:19:46.850 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T10:19:46.850 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T10:19:46.850 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T10:19:46.851 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T10:19:46.851 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T10:19:46.851 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T10:19:46.851 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T10:19:46.851 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T10:19:46.851 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T10:19:46.851 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T10:19:46.851 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T10:19:46.851 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T10:19:46.851 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T10:19:46.852 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T10:19:46.853 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T10:19:46.853 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T10:19:46.853 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T10:19:46.853 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T10:19:46.853 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T10:19:46.853 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T10:19:46.853 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T10:19:46.853 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T10:19:46.853 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T10:19:47.066 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T10:19:47.097 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T10:19:47.117 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T10:19:47.910 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T10:19:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T10:19:48.042 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T10:19:48.124 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T10:19:48.164 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T10:19:48.164 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T10:19:48.239 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T10:19:48.239 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T10:19:48.995 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T10:19:48.995 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:48.995 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-04-21T10:19:48.995 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.995 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.996 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-04-21T10:19:48.996 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T10:19:48.996 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T10:19:48.996 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T10:19:48.996 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T10:19:48.996 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.996 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.996 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.997 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.997 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:48.997 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.997 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.997 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.997 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:48.997 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:48.997 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:48.997 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:48.998 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:48.998 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.998 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.998 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:48.998 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.998 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.998 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.999 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:48.999 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:48.999 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.999 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:48.999 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:48.999 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T10:19:48.999 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T10:19:49.000 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T10:19:49.000 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T10:19:49.000 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T10:19:49.000 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T10:19:49.001 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T10:19:49.002 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T10:19:49.002 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T10:19:49.002 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T10:19:49.002 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T10:19:49.002 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.002 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.002 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.003 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T10:19:49.003 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T10:19:49.003 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T10:19:49.003 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T10:19:49.003 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T10:19:49.003 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.004 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.004 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T10:19:49.004 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.004 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T10:19:49.004 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T10:19:49.004 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T10:19:49.004 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T10:19:49.004 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T10:19:49.004 INFO:teuthology.orchestra.run.smithi026.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T10:19:49.004 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T10:19:49.005 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T10:19:49.006 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T10:19:49.006 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T10:19:49.006 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.006 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.006 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.006 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T10:19:49.006 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T10:19:49.006 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T10:19:49.006 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T10:19:49.007 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T10:19:49.007 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T10:19:49.008 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T10:19:49.008 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T10:19:49.008 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T10:19:49.008 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T10:19:49.008 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T10:19:49.008 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T10:19:49.008 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T10:19:49.008 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T10:19:49.008 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T10:19:49.009 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T10:19:49.009 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T10:19:49.009 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T10:19:49.009 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T10:19:49.009 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T10:19:49.009 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T10:19:49.010 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T10:19:49.011 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T10:19:49.011 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T10:19:49.011 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T10:19:49.011 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T10:19:49.011 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T10:19:49.011 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.011 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.011 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T10:19:49.011 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T10:19:49.011 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T10:19:49.011 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.012 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T10:19:49.013 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T10:19:49.013 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T10:19:49.013 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T10:19:49.013 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T10:19:49.013 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T10:19:49.013 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T10:19:49.014 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T10:19:49.014 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T10:19:49.014 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T10:19:49.014 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T10:19:49.014 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T10:19:49.014 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T10:19:49.014 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T10:19:49.014 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T10:19:49.014 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.014 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.014 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:49.015 INFO:teuthology.orchestra.run.smithi026.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T10:19:49.015 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T10:19:49.015 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T10:19:49.015 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T10:19:49.015 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T10:19:49.015 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T10:19:49.015 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T10:19:49.015 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:19:49.015 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-04-21T10:19:49.346 DEBUG:teuthology.parallel:result is None 2024-04-21T10:19:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T10:19:51.174 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T10:19:51.215 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T10:19:51.247 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T10:19:51.275 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T10:19:51.457 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T10:19:51.512 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T10:19:51.542 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T10:19:52.153 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T10:19:52.190 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T10:19:52.209 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T10:19:52.991 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T10:19:53.077 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T10:19:53.118 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T10:19:53.199 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T10:19:53.248 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T10:19:53.248 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T10:19:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T10:19:53.332 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T10:19:54.316 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T10:19:54.316 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T10:19:54.316 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T10:19:54.316 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T10:19:54.316 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T10:19:54.316 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T10:19:54.316 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T10:19:54.316 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T10:19:54.316 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T10:19:54.316 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T10:19:54.317 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T10:19:54.318 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T10:19:54.318 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T10:19:54.330 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T10:19:54.330 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T10:19:54.330 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T10:19:54.330 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T10:19:54.331 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T10:19:54.332 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T10:19:54.333 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T10:19:54.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T10:19:54.335 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T10:19:54.336 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T10:19:54.337 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T10:19:54.338 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T10:19:54.339 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T10:19:54.339 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T10:19:54.339 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T10:19:54.339 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T10:19:54.339 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T10:19:54.339 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T10:19:54.339 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T10:19:54.339 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T10:19:54.339 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T10:19:54.339 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T10:19:54.339 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T10:19:56.410 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T10:19:56.410 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.411 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.412 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T10:19:56.413 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.414 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T10:19:56.415 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T10:19:56.417 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T10:19:56.418 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.419 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T10:19:56.420 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T10:19:56.420 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T10:19:56.420 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.420 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T10:19:56.420 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T10:19:56.420 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T10:19:56.420 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T10:19:56.420 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T10:19:56.420 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T10:19:56.420 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T10:19:56.420 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T10:19:56.421 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T10:19:56.422 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T10:19:56.422 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T10:19:56.422 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T10:19:56.422 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-21T10:19:56.422 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-21T10:19:56.807 DEBUG:teuthology.parallel:result is None 2024-04-21T10:19:59.118 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T10:19:59.118 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T10:19:59.118 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T10:19:59.118 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T10:19:59.119 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T10:19:59.120 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T10:19:59.120 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T10:19:59.120 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T10:19:59.120 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T10:19:59.120 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T10:19:59.120 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T10:19:59.120 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T10:19:59.120 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T10:19:59.120 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T10:19:59.132 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T10:19:59.132 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T10:19:59.132 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T10:19:59.132 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T10:19:59.132 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T10:19:59.133 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T10:19:59.134 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T10:19:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T10:19:59.136 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T10:19:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T10:19:59.138 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T10:19:59.139 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T10:19:59.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T10:19:59.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T10:19:59.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T10:19:59.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T10:19:59.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T10:19:59.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T10:19:59.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T10:19:59.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T10:19:59.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T10:19:59.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T10:19:59.141 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T10:20:01.140 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T10:20:01.141 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:20:01.141 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2024-04-21T10:20:01.141 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.141 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.141 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-04-21T10:20:01.141 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T10:20:01.141 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T10:20:01.141 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T10:20:01.141 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T10:20:01.141 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.141 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.141 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:20:01.142 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T10:20:01.143 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T10:20:01.144 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T10:20:01.145 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T10:20:01.146 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T10:20:01.147 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T10:20:01.148 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T10:20:01.148 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T10:20:01.148 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T10:20:01.148 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T10:20:01.148 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T10:20:01.148 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T10:20:01.148 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T10:20:01.148 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T10:20:01.148 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T10:20:01.148 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T10:20:01.148 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T10:20:01.148 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T10:20:01.149 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T10:20:01.150 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.151 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.153 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T10:20:01.153 INFO:teuthology.orchestra.run.smithi190.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T10:20:01.153 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T10:20:01.153 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T10:20:01.153 INFO:teuthology.orchestra.run.smithi190.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T10:20:01.153 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T10:20:01.153 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T10:20:01.154 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T10:20:01.154 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:20:01.154 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-04-21T10:20:01.477 DEBUG:teuthology.parallel:result is None 2024-04-21T10:20:01.478 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T10:20:01.630 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T10:20:01.657 INFO:teuthology.orchestra.run.smithi026.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T10:20:01.657 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T10:20:01.658 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T10:20:01.659 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T10:20:01.806 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T10:20:01.831 INFO:teuthology.orchestra.run.smithi169.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T10:20:01.831 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T10:20:01.832 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T10:20:01.833 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T10:20:02.055 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T10:20:02.077 INFO:teuthology.orchestra.run.smithi190.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T10:20:02.077 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T10:20:02.077 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T10:20:02.079 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-21T10:20:02.080 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:20:02.080 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T10:20:02.124 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-21T10:20:02.124 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T10:20:02.156 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-21T10:20:02.156 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T10:20:02.183 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-21T10:20:02.183 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:20:02.183 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T10:20:02.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T10:20:02.280 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-21T10:20:02.280 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T10:20:02.308 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T10:20:02.376 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-21T10:20:02.376 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T10:20:02.400 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T10:20:02.463 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-21T10:20:02.464 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:20:02.464 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T10:20:02.491 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T10:20:02.557 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-21T10:20:02.557 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T10:20:02.586 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T10:20:02.653 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-21T10:20:02.653 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T10:20:02.676 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T10:20:02.739 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-21T10:20:02.739 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:20:02.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T10:20:02.764 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T10:20:02.830 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-21T10:20:02.830 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T10:20:02.858 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T10:20:02.928 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-21T10:20:02.928 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T10:20:02.950 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T10:20:03.018 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-21T10:20:03.131 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'subvols': {'create': 2}} 2024-04-21T10:20:03.132 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T10:20:03.133 INFO:tasks.cephadm:Cluster fsid is b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:20:03.134 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-21T10:20:03.134 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.26', 'mon.b': '172.21.15.169', 'mon.c': '172.21.15.190'} 2024-04-21T10:20:03.134 INFO:tasks.cephadm:First mon is mon.a on smithi026 2024-04-21T10:20:03.134 INFO:tasks.cephadm:First mgr is x 2024-04-21T10:20:03.134 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-21T10:20:03.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo hostname $(hostname -s) 2024-04-21T10:20:03.159 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname $(hostname -s) 2024-04-21T10:20:03.188 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname $(hostname -s) 2024-04-21T10:20:03.212 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-21T10:20:03.213 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T10:20:03.428 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78740/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3154-ga630765c', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3154.ga630765c'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-19 11:13:53.331234', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-21T10:20:03.654 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 a630765c1f93c6e61d9b40c4b76437611d5ef29e from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T10:20:03.655 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm 2024-04-21T10:20:03.656 DEBUG:teuthology.orchestra.run.smithi026:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T10:20:04.121 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 10:20 /home/ubuntu/cephtest/cephadm 2024-04-21T10:20:04.122 DEBUG:teuthology.orchestra.run.smithi169:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T10:20:04.698 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 10:20 /home/ubuntu/cephtest/cephadm 2024-04-21T10:20:04.699 DEBUG:teuthology.orchestra.run.smithi190:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T10:20:05.305 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 10:20 /home/ubuntu/cephtest/cephadm 2024-04-21T10:20:05.305 DEBUG:teuthology.orchestra.run.smithi026:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T10:20:05.325 DEBUG:teuthology.orchestra.run.smithi169:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T10:20:05.344 DEBUG:teuthology.orchestra.run.smithi190:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T10:20:05.379 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e on all hosts... 2024-04-21T10:20:05.380 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T10:20:05.382 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T10:20:05.387 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T10:20:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T10:20:06.990 INFO:teuthology.orchestra.run.smithi169.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T10:20:07.117 INFO:teuthology.orchestra.run.smithi190.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T10:20:36.766 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:20:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T10:20:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T10:20:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "repo_digests": [ 2024-04-21T10:20:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T10:20:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-21T10:20:36.767 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:20:50.048 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-21T10:20:50.049 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T10:20:50.049 INFO:teuthology.orchestra.run.smithi169.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T10:20:50.049 INFO:teuthology.orchestra.run.smithi169.stdout: "repo_digests": [ 2024-04-21T10:20:50.049 INFO:teuthology.orchestra.run.smithi169.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T10:20:50.049 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2024-04-21T10:20:50.049 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-21T10:20:56.776 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-04-21T10:20:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T10:20:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T10:20:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: "repo_digests": [ 2024-04-21T10:20:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T10:20:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-04-21T10:20:56.777 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-04-21T10:20:56.802 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph 2024-04-21T10:20:56.836 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph 2024-04-21T10:20:56.868 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph 2024-04-21T10:20:56.895 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /etc/ceph 2024-04-21T10:20:56.925 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /etc/ceph 2024-04-21T10:20:56.952 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /etc/ceph 2024-04-21T10:20:56.976 INFO:tasks.cephadm:Writing seed config... 2024-04-21T10:20:56.977 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-21T10:20:56.977 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-21T10:20:56.977 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-21T10:20:56.977 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-21T10:20:56.977 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-21T10:20:56.977 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-21T10:20:56.977 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-21T10:20:56.977 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-21T10:20:56.977 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-21T10:20:56.978 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-21T10:20:56.979 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-21T10:20:56.980 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-21T10:20:56.980 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-21T10:20:56.980 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-21T10:20:56.980 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-21T10:20:56.980 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-21T10:20:56.980 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-21T10:20:56.980 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-21T10:20:56.980 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-21T10:20:56.980 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-21T10:20:56.980 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-21T10:20:56.980 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-21T10:20:56.981 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:20:56.981 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-21T10:20:57.000 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b77245c8-ffc8-11ee-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-21T10:20:57.000 DEBUG:teuthology.orchestra.run.smithi026:mon.a> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.a.service 2024-04-21T10:20:57.043 DEBUG:teuthology.orchestra.run.smithi026:mgr.x> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.x.service 2024-04-21T10:20:57.086 INFO:tasks.cephadm:Bootstrapping... 2024-04-21T10:20:57.086 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e -v bootstrap --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.26 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-21T10:20:57.287 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-04-21T10:20:57.296 INFO:teuthology.orchestra.run.smithi026.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e', '-v', 'bootstrap', '--fsid', 'b77245c8-ffc8-11ee-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.26', '--skip-admin-label'] 2024-04-21T10:20:57.297 INFO:teuthology.orchestra.run.smithi026.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-21T10:20:57.297 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying podman|docker is present... 2024-04-21T10:20:57.314 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-21T10:20:57.314 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying lvm2 is present... 2024-04-21T10:20:57.314 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying time synchronization is in place... 2024-04-21T10:20:57.321 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-21T10:20:57.321 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-21T10:20:57.327 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-21T10:20:57.327 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-04-21T10:20:57.334 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout enabled 2024-04-21T10:20:57.340 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout active 2024-04-21T10:20:57.340 INFO:teuthology.orchestra.run.smithi026.stdout:Unit chronyd.service is enabled and running 2024-04-21T10:20:57.340 INFO:teuthology.orchestra.run.smithi026.stdout:Repeating the final host check... 2024-04-21T10:20:57.365 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-21T10:20:57.365 INFO:teuthology.orchestra.run.smithi026.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-21T10:20:57.365 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl is present 2024-04-21T10:20:57.365 INFO:teuthology.orchestra.run.smithi026.stdout:lvcreate is present 2024-04-21T10:20:57.371 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-21T10:20:57.371 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-21T10:20:57.377 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-21T10:20:57.378 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-04-21T10:20:57.384 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout enabled 2024-04-21T10:20:57.390 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout active 2024-04-21T10:20:57.390 INFO:teuthology.orchestra.run.smithi026.stdout:Unit chronyd.service is enabled and running 2024-04-21T10:20:57.390 INFO:teuthology.orchestra.run.smithi026.stdout:Host looks OK 2024-04-21T10:20:57.390 INFO:teuthology.orchestra.run.smithi026.stdout:Cluster fsid: b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:20:57.390 INFO:teuthology.orchestra.run.smithi026.stdout:Acquiring lock 140457759050912 on /run/cephadm/b77245c8-ffc8-11ee-bc93-c7b262605968.lock 2024-04-21T10:20:57.390 INFO:teuthology.orchestra.run.smithi026.stdout:Lock 140457759050912 acquired on /run/cephadm/b77245c8-ffc8-11ee-bc93-c7b262605968.lock 2024-04-21T10:20:57.391 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying IP 172.21.15.26 port 3300 ... 2024-04-21T10:20:57.391 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying IP 172.21.15.26 port 6789 ... 2024-04-21T10:20:57.391 INFO:teuthology.orchestra.run.smithi026.stdout:Base mon IP(s) is [172.21.15.26:3300, 172.21.15.26:6789], mon addrv is [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2024-04-21T10:20:57.394 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.26 metric 100 2024-04-21T10:20:57.394 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.26 metric 100 2024-04-21T10:20:57.397 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-21T10:20:57.397 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-21T10:20:57.397 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-21T10:20:57.399 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-21T10:20:57.400 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-21T10:20:57.400 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-21T10:20:57.400 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-21T10:20:57.400 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-21T10:20:57.400 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-21T10:20:57.400 INFO:teuthology.orchestra.run.smithi026.stdout:Mon IP `172.21.15.26` is in CIDR network `172.21.0.0/20` 2024-04-21T10:20:57.401 INFO:teuthology.orchestra.run.smithi026.stdout:Mon IP `172.21.15.26` is in CIDR network `172.21.0.0/20` 2024-04-21T10:20:57.401 INFO:teuthology.orchestra.run.smithi026.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-21T10:20:57.401 INFO:teuthology.orchestra.run.smithi026.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-21T10:20:57.402 INFO:teuthology.orchestra.run.smithi026.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T10:20:57.854 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 2024-04-21T10:20:57.854 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T10:20:57.854 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Getting image source signatures 2024-04-21T10:20:57.854 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying blob sha256:7eab8a2da10d06ac88aea1fd8998a2559b4de9d07c585e3b1e6ee6c4d35fc119 2024-04-21T10:20:57.854 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-21T10:20:57.854 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying config sha256:b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 2024-04-21T10:20:57.854 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-21T10:20:58.794 INFO:teuthology.orchestra.run.smithi026.stdout:ceph: stdout ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T10:20:58.794 INFO:teuthology.orchestra.run.smithi026.stdout:Ceph version: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T10:20:58.794 INFO:teuthology.orchestra.run.smithi026.stdout:Extracting ceph user uid/gid from container image... 2024-04-21T10:20:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:stat: stdout 167 167 2024-04-21T10:20:59.650 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial keys... 2024-04-21T10:21:00.499 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQCL6CRm11TcOBAAz5hTOJZrwKk0Cv+Z1cLYKA== 2024-04-21T10:21:04.628 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQCN6CRmX7jYIRAA45VOd6JXlLaTe/d9lolYxQ== 2024-04-21T10:21:05.437 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQCQ6CRm/e9nOBAAZc0svHCXcmQAHk38aBcPcg== 2024-04-21T10:21:05.438 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial monmap... 2024-04-21T10:21:06.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-21T10:21:06.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-21T10:21:06.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:21:06.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-21T10:21:06.299 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool for a [v2:172.21.15.26:3300,v1:172.21.15.26:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-21T10:21:06.299 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = quincy 2024-04-21T10:21:06.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: set fsid to b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:21:06.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-21T10:21:06.300 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:21:06.300 INFO:teuthology.orchestra.run.smithi026.stdout:Creating mon... 2024-04-21T10:21:07.175 INFO:teuthology.orchestra.run.smithi026.stdout:create mon.a on 2024-04-21T10:21:07.391 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-21T10:21:07.544 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-21T10:21:07.735 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b77245c8-ffc8-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-b77245c8-ffc8-11ee-bc93-c7b262605968.target. 2024-04-21T10:21:07.735 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b77245c8-ffc8-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-b77245c8-ffc8-11ee-bc93-c7b262605968.target. 2024-04-21T10:21:07.979 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.a 2024-04-21T10:21:07.980 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.a.service: Unit ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.a.service not loaded. 2024-04-21T10:21:08.145 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b77245c8-ffc8-11ee-bc93-c7b262605968.target.wants/ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@.service. 2024-04-21T10:21:08.666 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T10:21:08.666 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-04-21T10:21:08.672 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T10:21:08.672 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-04-21T10:21:08.673 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-04-21T10:21:08.673 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to enable service . firewalld.service is not available 2024-04-21T10:21:08.673 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mon to start... 2024-04-21T10:21:08.673 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mon... 2024-04-21T10:21:10.089 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout cluster: 2024-04-21T10:21:10.089 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout id: b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:21:10.089 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout services: 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.543213s) 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout data: 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout pgs: 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:10.090 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.098+0000 7f0aabf42700 1 Processor -- start 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.099+0000 7f0aabf42700 1 -- start start 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.099+0000 7f0aabf42700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 0x7f0aa4108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.099+0000 7f0aabf42700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4109060 con 0x7f0aa4108670 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.099+0000 7f0aa9cde700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 0x7f0aa4108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.099+0000 7f0aa9cde700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 0x7f0aa4108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45652/0 (socket says 172.21.15.26:45652) 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.099+0000 7f0aa9cde700 1 -- 172.21.15.26:0/169276996 learned_addr learned my addr 172.21.15.26:0/169276996 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.099+0000 7f0aa9cde700 1 -- 172.21.15.26:0/169276996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa4109820 con 0x7f0aa4108670 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.100+0000 7f0aa9cde700 1 --2- 172.21.15.26:0/169276996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 0x7f0aa4108a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0a940098a0 tx=0x7f0a94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa40d769851321a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.100+0000 7f0aa8cdc700 1 -- 172.21.15.26:0/169276996 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a94004160 con 0x7f0aa4108670 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.100+0000 7f0aa8cdc700 1 -- 172.21.15.26:0/169276996 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a94004300 con 0x7f0aa4108670 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.100+0000 7f0aa8cdc700 1 -- 172.21.15.26:0/169276996 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a940036a0 con 0x7f0aa4108670 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.100+0000 7f0aabf42700 1 -- 172.21.15.26:0/169276996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 msgr2=0x7f0aa4108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:10.091 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.100+0000 7f0aabf42700 1 --2- 172.21.15.26:0/169276996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 0x7f0aa4108a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0a940098a0 tx=0x7f0a94009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.100+0000 7f0aabf42700 1 -- 172.21.15.26:0/169276996 shutdown_connections 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.100+0000 7f0aabf42700 1 --2- 172.21.15.26:0/169276996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 0x7f0aa4108a90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.100+0000 7f0aabf42700 1 -- 172.21.15.26:0/169276996 >> 172.21.15.26:0/169276996 conn(0x7f0aa4103c30 msgr2=0x7f0aa4106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.101+0000 7f0aabf42700 1 -- 172.21.15.26:0/169276996 shutdown_connections 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.101+0000 7f0aabf42700 1 -- 172.21.15.26:0/169276996 wait complete. 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.101+0000 7f0aabf42700 1 Processor -- start 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.101+0000 7f0aabf42700 1 -- start start 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.101+0000 7f0aabf42700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 0x7f0aa407eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.101+0000 7f0aabf42700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4109d50 con 0x7f0aa4108670 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.102+0000 7f0aa9cde700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 0x7f0aa407eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.102+0000 7f0aa9cde700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 0x7f0aa407eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45664/0 (socket says 172.21.15.26:45664) 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.102+0000 7f0aa9cde700 1 -- 172.21.15.26:0/2271199112 learned_addr learned my addr 172.21.15.26:0/2271199112 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:10.092 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.102+0000 7f0aa9cde700 1 -- 172.21.15.26:0/2271199112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa407f0b0 con 0x7f0aa4108670 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.102+0000 7f0aa9cde700 1 --2- 172.21.15.26:0/2271199112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 0x7f0aa407eb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0a94000c00 tx=0x7f0a94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.102+0000 7f0a9affd700 1 -- 172.21.15.26:0/2271199112 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a94003db0 con 0x7f0aa4108670 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.102+0000 7f0a9affd700 1 -- 172.21.15.26:0/2271199112 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a94011630 con 0x7f0aa4108670 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.102+0000 7f0aabf42700 1 -- 172.21.15.26:0/2271199112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa407d2a0 con 0x7f0aa4108670 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.102+0000 7f0a9affd700 1 -- 172.21.15.26:0/2271199112 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a94011950 con 0x7f0aa4108670 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.102+0000 7f0aabf42700 1 -- 172.21.15.26:0/2271199112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa407d7e0 con 0x7f0aa4108670 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.102+0000 7f0a9affd700 1 -- 172.21.15.26:0/2271199112 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0a9401a460 con 0x7f0aa4108670 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.102+0000 7f0a9affd700 1 -- 172.21.15.26:0/2271199112 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0a9401b780 con 0x7f0aa4108670 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.103+0000 7f0aabf42700 1 -- 172.21.15.26:0/2271199112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa40640a0 con 0x7f0aa4108670 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.106+0000 7f0a9affd700 1 -- 172.21.15.26:0/2271199112 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0a94031020 con 0x7f0aa4108670 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.179+0000 7f0aabf42700 1 -- 172.21.15.26:0/2271199112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f0aa41955f0 con 0x7f0aa4108670 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.180+0000 7f0a9affd700 1 -- 172.21.15.26:0/2271199112 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f0a9401b9d0 con 0x7f0aa4108670 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.181+0000 7f0aabf42700 1 -- 172.21.15.26:0/2271199112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 msgr2=0x7f0aa407eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:10.093 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.181+0000 7f0aabf42700 1 --2- 172.21.15.26:0/2271199112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 0x7f0aa407eb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0a94000c00 tx=0x7f0a94004750 comp rx=0 tx=0).stop 2024-04-21T10:21:10.094 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.181+0000 7f0aabf42700 1 -- 172.21.15.26:0/2271199112 shutdown_connections 2024-04-21T10:21:10.094 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.181+0000 7f0aabf42700 1 --2- 172.21.15.26:0/2271199112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aa4108670 0x7f0aa407eb70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:10.094 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.181+0000 7f0aabf42700 1 -- 172.21.15.26:0/2271199112 >> 172.21.15.26:0/2271199112 conn(0x7f0aa4103c30 msgr2=0x7f0aa4192930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:10.094 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.181+0000 7f0aabf42700 1 -- 172.21.15.26:0/2271199112 shutdown_connections 2024-04-21T10:21:10.094 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:09.181+0000 7f0aabf42700 1 -- 172.21.15.26:0/2271199112 wait complete. 2024-04-21T10:21:10.094 INFO:teuthology.orchestra.run.smithi026.stdout:mon is available 2024-04-21T10:21:10.094 INFO:teuthology.orchestra.run.smithi026.stdout:Assimilating anything we can from ceph.conf... 2024-04-21T10:21:11.109 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:11.109 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [global] 2024-04-21T10:21:11.109 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout fsid = b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:21:11.109 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-21T10:21:11.109 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2024-04-21T10:21:11.109 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-21T10:21:11.109 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-21T10:21:11.109 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-21T10:21:11.109 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-21T10:21:11.109 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:11.109 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [osd] 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.490+0000 7f8ac0f32700 1 Processor -- start 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.490+0000 7f8ac0f32700 1 -- start start 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.490+0000 7f8ac0f32700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 0x7f8abc10b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.490+0000 7f8ac0f32700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc07ce50 con 0x7f8abc1090e0 2024-04-21T10:21:11.111 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.491+0000 7f8aba59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 0x7f8abc10b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:11.111 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.491+0000 7f8aba59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 0x7f8abc10b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45678/0 (socket says 172.21.15.26:45678) 2024-04-21T10:21:11.111 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.491+0000 7f8aba59c700 1 -- 172.21.15.26:0/3033562081 learned_addr learned my addr 172.21.15.26:0/3033562081 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:11.111 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.491+0000 7f8aba59c700 1 -- 172.21.15.26:0/3033562081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc10ba10 con 0x7f8abc1090e0 2024-04-21T10:21:11.111 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.491+0000 7f8aba59c700 1 --2- 172.21.15.26:0/3033562081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 0x7f8abc10b4d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0098a0 tx=0x7f8aac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8a7c8027c194da server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:11.111 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.491+0000 7f8ab959a700 1 -- 172.21.15.26:0/3033562081 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aac004160 con 0x7f8abc1090e0 2024-04-21T10:21:11.111 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.491+0000 7f8ab959a700 1 -- 172.21.15.26:0/3033562081 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aac004300 con 0x7f8abc1090e0 2024-04-21T10:21:11.111 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.491+0000 7f8ab959a700 1 -- 172.21.15.26:0/3033562081 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aac0036a0 con 0x7f8abc1090e0 2024-04-21T10:21:11.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.492+0000 7f8ac0f32700 1 -- 172.21.15.26:0/3033562081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 msgr2=0x7f8abc10b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:11.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.492+0000 7f8ac0f32700 1 --2- 172.21.15.26:0/3033562081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 0x7f8abc10b4d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0098a0 tx=0x7f8aac009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:11.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.492+0000 7f8ac0f32700 1 -- 172.21.15.26:0/3033562081 shutdown_connections 2024-04-21T10:21:11.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.492+0000 7f8ac0f32700 1 --2- 172.21.15.26:0/3033562081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 0x7f8abc10b4d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:11.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.492+0000 7f8ac0f32700 1 -- 172.21.15.26:0/3033562081 >> 172.21.15.26:0/3033562081 conn(0x7f8abc102ad0 msgr2=0x7f8abc104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:11.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.492+0000 7f8ac0f32700 1 -- 172.21.15.26:0/3033562081 shutdown_connections 2024-04-21T10:21:11.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.492+0000 7f8ac0f32700 1 -- 172.21.15.26:0/3033562081 wait complete. 2024-04-21T10:21:11.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.493+0000 7f8ac0f32700 1 Processor -- start 2024-04-21T10:21:11.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.493+0000 7f8ac0f32700 1 -- start start 2024-04-21T10:21:11.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.493+0000 7f8ac0f32700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 0x7f8abc19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:11.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.493+0000 7f8ac0f32700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc10be80 con 0x7f8abc1090e0 2024-04-21T10:21:11.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.493+0000 7f8aba59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 0x7f8abc19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.493+0000 7f8aba59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 0x7f8abc19bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45690/0 (socket says 172.21.15.26:45690) 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.493+0000 7f8aba59c700 1 -- 172.21.15.26:0/694503153 learned_addr learned my addr 172.21.15.26:0/694503153 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.493+0000 7f8aba59c700 1 -- 172.21.15.26:0/694503153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc19c190 con 0x7f8abc1090e0 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.493+0000 7f8aba59c700 1 --2- 172.21.15.26:0/694503153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 0x7f8abc19bc50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8aac000c00 tx=0x7f8aac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.494+0000 7f8aab7fe700 1 -- 172.21.15.26:0/694503153 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aac003db0 con 0x7f8abc1090e0 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.494+0000 7f8aab7fe700 1 -- 172.21.15.26:0/694503153 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aac011630 con 0x7f8abc1090e0 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.494+0000 7f8ac0f32700 1 -- 172.21.15.26:0/694503153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc19c420 con 0x7f8abc1090e0 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.494+0000 7f8aab7fe700 1 -- 172.21.15.26:0/694503153 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aac011950 con 0x7f8abc1090e0 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.494+0000 7f8ac0f32700 1 -- 172.21.15.26:0/694503153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc19f110 con 0x7f8abc1090e0 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.494+0000 7f8aab7fe700 1 -- 172.21.15.26:0/694503153 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8aac011af0 con 0x7f8abc1090e0 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.494+0000 7f8aab7fe700 1 -- 172.21.15.26:0/694503153 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8aac01c450 con 0x7f8abc1090e0 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.495+0000 7f8ac0f32700 1 -- 172.21.15.26:0/694503153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8abc0640a0 con 0x7f8abc1090e0 2024-04-21T10:21:11.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.497+0000 7f8aab7fe700 1 -- 172.21.15.26:0/694503153 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8aac032020 con 0x7f8abc1090e0 2024-04-21T10:21:11.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.567+0000 7f8ac0f32700 1 -- 172.21.15.26:0/694503153 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f8abc02e180 con 0x7f8abc1090e0 2024-04-21T10:21:11.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.570+0000 7f8aab7fe700 1 -- 172.21.15.26:0/694503153 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f8aac004260 con 0x7f8abc1090e0 2024-04-21T10:21:11.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.571+0000 7f8aab7fe700 1 -- 172.21.15.26:0/694503153 <== mon.0 v2:172.21.15.26:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8aac01c6a0 con 0x7f8abc1090e0 2024-04-21T10:21:11.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.571+0000 7f8ac0f32700 1 -- 172.21.15.26:0/694503153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 msgr2=0x7f8abc19bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:11.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.571+0000 7f8ac0f32700 1 --2- 172.21.15.26:0/694503153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 0x7f8abc19bc50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8aac000c00 tx=0x7f8aac004750 comp rx=0 tx=0).stop 2024-04-21T10:21:11.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.571+0000 7f8ac0f32700 1 -- 172.21.15.26:0/694503153 shutdown_connections 2024-04-21T10:21:11.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.571+0000 7f8ac0f32700 1 --2- 172.21.15.26:0/694503153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1090e0 0x7f8abc19bc50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:11.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.571+0000 7f8ac0f32700 1 -- 172.21.15.26:0/694503153 >> 172.21.15.26:0/694503153 conn(0x7f8abc102ad0 msgr2=0x7f8abc104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:11.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.571+0000 7f8ac0f32700 1 -- 172.21.15.26:0/694503153 shutdown_connections 2024-04-21T10:21:11.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:10.571+0000 7f8ac0f32700 1 -- 172.21.15.26:0/694503153 wait complete. 2024-04-21T10:21:11.114 INFO:teuthology.orchestra.run.smithi026.stdout:Generating new minimal ceph.conf... 2024-04-21T10:21:11.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:11 smithi026 ceph-mon[27594]: from='client.? 172.21.15.26:0/2258505435' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:21:12.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.545+0000 7f80e7b94700 1 Processor -- start 2024-04-21T10:21:12.201 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.545+0000 7f80e7b94700 1 -- start start 2024-04-21T10:21:12.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.545+0000 7f80e7b94700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 0x7f80e007d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:12.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.545+0000 7f80e7b94700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f80e007d820 con 0x7f80e007ee30 2024-04-21T10:21:12.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.546+0000 7f80e5930700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 0x7f80e007d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:12.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.546+0000 7f80e5930700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 0x7f80e007d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45694/0 (socket says 172.21.15.26:45694) 2024-04-21T10:21:12.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.546+0000 7f80e5930700 1 -- 172.21.15.26:0/1196505625 learned_addr learned my addr 172.21.15.26:0/1196505625 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:12.202 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.546+0000 7f80e5930700 1 -- 172.21.15.26:0/1196505625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e007d9a0 con 0x7f80e007ee30 2024-04-21T10:21:12.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.546+0000 7f80e5930700 1 --2- 172.21.15.26:0/1196505625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 0x7f80e007d2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0098a0 tx=0x7f80dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7565acaff6c3bb8a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:12.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.546+0000 7f80e492e700 1 -- 172.21.15.26:0/1196505625 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80dc004160 con 0x7f80e007ee30 2024-04-21T10:21:12.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.546+0000 7f80e492e700 1 -- 172.21.15.26:0/1196505625 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f80dc004300 con 0x7f80e007ee30 2024-04-21T10:21:12.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.546+0000 7f80e492e700 1 -- 172.21.15.26:0/1196505625 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80dc011af0 con 0x7f80e007ee30 2024-04-21T10:21:12.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.547+0000 7f80e7b94700 1 -- 172.21.15.26:0/1196505625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 msgr2=0x7f80e007d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:12.203 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.547+0000 7f80e7b94700 1 --2- 172.21.15.26:0/1196505625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 0x7f80e007d2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0098a0 tx=0x7f80dc009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:12.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.547+0000 7f80e7b94700 1 -- 172.21.15.26:0/1196505625 shutdown_connections 2024-04-21T10:21:12.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.547+0000 7f80e7b94700 1 --2- 172.21.15.26:0/1196505625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 0x7f80e007d2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:12.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.547+0000 7f80e7b94700 1 -- 172.21.15.26:0/1196505625 >> 172.21.15.26:0/1196505625 conn(0x7f80e0103c30 msgr2=0x7f80e0106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:12.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.547+0000 7f80e7b94700 1 -- 172.21.15.26:0/1196505625 shutdown_connections 2024-04-21T10:21:12.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.547+0000 7f80e7b94700 1 -- 172.21.15.26:0/1196505625 wait complete. 2024-04-21T10:21:12.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.548+0000 7f80e7b94700 1 Processor -- start 2024-04-21T10:21:12.204 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.548+0000 7f80e7b94700 1 -- start start 2024-04-21T10:21:12.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.548+0000 7f80e7b94700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 0x7f80e019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:12.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.548+0000 7f80e7b94700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f80e010eb20 con 0x7f80e007ee30 2024-04-21T10:21:12.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.548+0000 7f80e5930700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 0x7f80e019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:12.205 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.548+0000 7f80e5930700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 0x7f80e019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45710/0 (socket says 172.21.15.26:45710) 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.548+0000 7f80e5930700 1 -- 172.21.15.26:0/2258505435 learned_addr learned my addr 172.21.15.26:0/2258505435 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.548+0000 7f80e5930700 1 -- 172.21.15.26:0/2258505435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e019ec60 con 0x7f80e007ee30 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.548+0000 7f80e5930700 1 --2- 172.21.15.26:0/2258505435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 0x7f80e019e720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009fb0 tx=0x7f80dc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.549+0000 7f80d6ffd700 1 -- 172.21.15.26:0/2258505435 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80dc0036a0 con 0x7f80e007ee30 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.549+0000 7f80d6ffd700 1 -- 172.21.15.26:0/2258505435 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f80dc01a430 con 0x7f80e007ee30 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.549+0000 7f80e7b94700 1 -- 172.21.15.26:0/2258505435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e019eef0 con 0x7f80e007ee30 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.549+0000 7f80d6ffd700 1 -- 172.21.15.26:0/2258505435 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80dc019550 con 0x7f80e007ee30 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.549+0000 7f80e7b94700 1 -- 172.21.15.26:0/2258505435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e01a1be0 con 0x7f80e007ee30 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.549+0000 7f80d6ffd700 1 -- 172.21.15.26:0/2258505435 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f80dc019b90 con 0x7f80e007ee30 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.549+0000 7f80d6ffd700 1 -- 172.21.15.26:0/2258505435 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f80dc024730 con 0x7f80e007ee30 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.550+0000 7f80e7b94700 1 -- 172.21.15.26:0/2258505435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e00640a0 con 0x7f80e007ee30 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.552+0000 7f80d6ffd700 1 -- 172.21.15.26:0/2258505435 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f80dc043e50 con 0x7f80e007ee30 2024-04-21T10:21:12.206 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.622+0000 7f80e7b94700 1 -- 172.21.15.26:0/2258505435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f80e01a2130 con 0x7f80e007ee30 2024-04-21T10:21:12.207 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.623+0000 7f80d6ffd700 1 -- 172.21.15.26:0/2258505435 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f80dc032030 con 0x7f80e007ee30 2024-04-21T10:21:12.207 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.624+0000 7f80e7b94700 1 -- 172.21.15.26:0/2258505435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 msgr2=0x7f80e019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:12.207 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.624+0000 7f80e7b94700 1 --2- 172.21.15.26:0/2258505435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 0x7f80e019e720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009fb0 tx=0x7f80dc011940 comp rx=0 tx=0).stop 2024-04-21T10:21:12.207 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.624+0000 7f80e7b94700 1 -- 172.21.15.26:0/2258505435 shutdown_connections 2024-04-21T10:21:12.207 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.624+0000 7f80e7b94700 1 --2- 172.21.15.26:0/2258505435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80e007ee30 0x7f80e019e720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:12.207 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.624+0000 7f80e7b94700 1 -- 172.21.15.26:0/2258505435 >> 172.21.15.26:0/2258505435 conn(0x7f80e0103c30 msgr2=0x7f80e01047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:12.207 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.624+0000 7f80e7b94700 1 -- 172.21.15.26:0/2258505435 shutdown_connections 2024-04-21T10:21:12.207 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:11.624+0000 7f80e7b94700 1 -- 172.21.15.26:0/2258505435 wait complete. 2024-04-21T10:21:12.207 INFO:teuthology.orchestra.run.smithi026.stdout:Restarting the monitor... 2024-04-21T10:21:12.463 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:12 smithi026 systemd[1]: Stopping Ceph mon.a for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:21:12.746 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:12 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a[27590]: 2024-04-21T10:21:12.462+0000 7f806d805700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:21:12.746 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:12 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a[27590]: 2024-04-21T10:21:12.462+0000 7f806d805700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-21T10:21:12.746 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:12 smithi026 podman[27838]: 2024-04-21 10:21:12.504122761 +0000 UTC m=+0.139363612 container died 80a7e89ed4120159983248f2b433c1c0544b283f5f2921102b01052435e0bc1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-21T10:21:13.012 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:12 smithi026 podman[27838]: 2024-04-21 10:21:12.837568973 +0000 UTC m=+0.472809812 container cleanup 80a7e89ed4120159983248f2b433c1c0544b283f5f2921102b01052435e0bc1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux ) 2024-04-21T10:21:13.013 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:12 smithi026 bash[27838]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a 2024-04-21T10:21:13.262 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 podman[27853]: 2024-04-21 10:21:13.013490723 +0000 UTC m=+0.505151210 container remove 80a7e89ed4120159983248f2b433c1c0544b283f5f2921102b01052435e0bc1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, ceph=True) 2024-04-21T10:21:13.570 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 systemd[1]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-21T10:21:13.570 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 systemd[1]: Stopped Ceph mon.a for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:21:13.570 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 systemd[1]: Starting Ceph mon.a for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:21:13.570 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 podman[27946]: 2024-04-21 10:21:13.383858599 +0000 UTC m=+0.018524825 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T10:21:13.571 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 podman[27946]: 2024-04-21 10:21:13.481016263 +0000 UTC m=+0.115682477 container create 1526264d8981cbd07a9d45e7c0d71bc634e1809f178ac78a012ae3044bff604f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T10:21:13.748 INFO:teuthology.orchestra.run.smithi026.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-21T10:21:13.873 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 podman[27946]: 2024-04-21 10:21:13.66919486 +0000 UTC m=+0.303861069 container init 1526264d8981cbd07a9d45e7c0d71bc634e1809f178ac78a012ae3044bff604f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-21T10:21:13.873 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 podman[27946]: 2024-04-21 10:21:13.672216914 +0000 UTC m=+0.306883121 container start 1526264d8981cbd07a9d45e7c0d71bc634e1809f178ac78a012ae3044bff604f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:21:13.873 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: set uid:gid to 167:167 (ceph:ceph) 2024-04-21T10:21:13.873 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev), process ceph-mon, pid 2 2024-04-21T10:21:13.873 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: pidfile_write: ignore empty --pid-file 2024-04-21T10:21:13.873 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: load: jerasure load: lrc 2024-04-21T10:21:13.873 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: RocksDB version: 7.9.2 2024-04-21T10:21:13.873 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Git sha 0 2024-04-21T10:21:13.873 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Compile date 2024-04-19 14:19:22 2024-04-21T10:21:13.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: DB SUMMARY 2024-04-21T10:21:13.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: DB Session ID: S27ZS24ZV5LOBJSER52N 2024-04-21T10:21:13.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: CURRENT file: CURRENT 2024-04-21T10:21:13.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: IDENTITY file: IDENTITY 2024-04-21T10:21:13.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-21T10:21:13.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-21T10:21:13.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106024 ; 2024-04-21T10:21:13.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.error_if_exists: 0 2024-04-21T10:21:13.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.create_if_missing: 0 2024-04-21T10:21:13.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.paranoid_checks: 1 2024-04-21T10:21:13.874 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-21T10:21:13.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-21T10:21:13.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-21T10:21:13.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.env: 0x5648ac650cc0 2024-04-21T10:21:13.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.fs: PosixFileSystem 2024-04-21T10:21:13.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.info_log: 0x5648ad439be0 2024-04-21T10:21:13.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_file_opening_threads: 16 2024-04-21T10:21:13.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.statistics: (nil) 2024-04-21T10:21:13.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.use_fsync: 0 2024-04-21T10:21:13.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_log_file_size: 0 2024-04-21T10:21:13.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-21T10:21:13.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-21T10:21:13.875 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.keep_log_file_num: 1000 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.recycle_log_file_num: 0 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.allow_fallocate: 1 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.allow_mmap_reads: 0 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.allow_mmap_writes: 0 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.use_direct_reads: 0 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.create_missing_column_families: 0 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.db_log_dir: 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.wal_dir: 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-21T10:21:13.876 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-21T10:21:13.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-21T10:21:13.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-21T10:21:13.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.advise_random_on_open: 1 2024-04-21T10:21:13.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.db_write_buffer_size: 0 2024-04-21T10:21:13.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.write_buffer_manager: 0x5648ad3cd0e0 2024-04-21T10:21:13.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-21T10:21:13.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-21T10:21:13.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-21T10:21:13.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.rate_limiter: (nil) 2024-04-21T10:21:13.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-21T10:21:13.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.wal_recovery_mode: 2 2024-04-21T10:21:13.877 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.enable_thread_tracking: 0 2024-04-21T10:21:13.878 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.enable_pipelined_write: 0 2024-04-21T10:21:13.878 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.unordered_write: 0 2024-04-21T10:21:13.878 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-21T10:21:13.878 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-21T10:21:13.878 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-21T10:21:13.878 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-21T10:21:13.878 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.row_cache: None 2024-04-21T10:21:13.878 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.wal_filter: None 2024-04-21T10:21:13.880 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-21T10:21:13.880 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.allow_ingest_behind: 0 2024-04-21T10:21:13.880 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.two_write_queues: 0 2024-04-21T10:21:13.880 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.manual_wal_flush: 0 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.wal_compression: 0 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.atomic_flush: 0 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.log_readahead_size: 0 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.best_efforts_recovery: 0 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.allow_data_in_errors: 0 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.db_host_id: __hostname__ 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-21T10:21:13.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_background_jobs: 2 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_background_compactions: -1 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_subcompactions: 1 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_total_wal_size: 0 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_open_files: -1 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bytes_per_sync: 0 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-21T10:21:13.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_readahead_size: 0 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_background_flushes: -1 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Compression algorithms supported: 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: kZSTD supported: 0 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: kXpressCompression supported: 0 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: kLZ4HCCompression supported: 1 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: kZlibCompression supported: 1 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: kSnappyCompression supported: 1 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: kLZ4Compression supported: 1 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: kBZip2Compression supported: 0 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-21T10:21:13.883 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.merge_operator: 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_filter: None 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_filter_factory: None 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.sst_partitioner_factory: None 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5648ad439dc0) 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout: cache_index_and_filter_blocks: 1 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout: pin_top_level_index_and_filter: 1 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_type: 0 2024-04-21T10:21:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout: data_block_index_type: 0 2024-04-21T10:21:13.885 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_shortening: 1 2024-04-21T10:21:13.885 INFO:journalctl@ceph.mon.a.smithi026.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-21T10:21:13.885 INFO:journalctl@ceph.mon.a.smithi026.stdout: checksum: 4 2024-04-21T10:21:13.885 INFO:journalctl@ceph.mon.a.smithi026.stdout: no_block_cache: 0 2024-04-21T10:21:13.885 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache: 0x5648ad3ff1f0 2024-04-21T10:21:13.885 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_name: BinnedLRUCache 2024-04-21T10:21:13.885 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_options: 2024-04-21T10:21:13.886 INFO:journalctl@ceph.mon.a.smithi026.stdout: capacity : 536870912 2024-04-21T10:21:13.886 INFO:journalctl@ceph.mon.a.smithi026.stdout: num_shard_bits : 4 2024-04-21T10:21:13.886 INFO:journalctl@ceph.mon.a.smithi026.stdout: strict_capacity_limit : 0 2024-04-21T10:21:13.886 INFO:journalctl@ceph.mon.a.smithi026.stdout: high_pri_pool_ratio: 0.000 2024-04-21T10:21:13.886 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_compressed: (nil) 2024-04-21T10:21:13.886 INFO:journalctl@ceph.mon.a.smithi026.stdout: persistent_cache: (nil) 2024-04-21T10:21:13.886 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_size: 4096 2024-04-21T10:21:13.886 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_size_deviation: 10 2024-04-21T10:21:13.886 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_restart_interval: 16 2024-04-21T10:21:13.886 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_block_restart_interval: 1 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: metadata_block_size: 4096 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: partition_filters: 0 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: use_delta_encoding: 1 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: filter_policy: bloomfilter 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: whole_key_filtering: 1 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: verify_compression: 0 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: read_amp_bytes_per_bit: 0 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: format_version: 5 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: enable_index_compression: 1 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_align: 0 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: max_auto_readahead_size: 262144 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: prepopulate_block_cache: 0 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: initial_auto_readahead_size: 8192 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout: num_file_reads_for_auto_readahead: 2 2024-04-21T10:21:13.887 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.write_buffer_size: 33554432 2024-04-21T10:21:13.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_write_buffer_number: 2 2024-04-21T10:21:13.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compression: NoCompression 2024-04-21T10:21:13.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bottommost_compression: Disabled 2024-04-21T10:21:13.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.prefix_extractor: nullptr 2024-04-21T10:21:13.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-21T10:21:13.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.num_levels: 7 2024-04-21T10:21:13.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-21T10:21:13.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-21T10:21:13.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-21T10:21:13.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-21T10:21:13.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-21T10:21:13.888 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-21T10:21:13.889 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-21T10:21:13.889 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-21T10:21:13.889 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-21T10:21:13.889 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-21T10:21:13.889 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-21T10:21:13.889 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-21T10:21:13.889 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-21T10:21:13.889 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compression_opts.level: 32767 2024-04-21T10:21:13.889 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compression_opts.strategy: 0 2024-04-21T10:21:13.890 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-21T10:21:13.890 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-21T10:21:13.890 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-21T10:21:13.890 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-21T10:21:13.890 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compression_opts.enabled: false 2024-04-21T10:21:13.890 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-21T10:21:13.890 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-21T10:21:13.890 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-21T10:21:13.890 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-21T10:21:13.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.target_file_size_base: 67108864 2024-04-21T10:21:13.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-21T10:21:13.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-21T10:21:13.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-21T10:21:13.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-21T10:21:13.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-21T10:21:13.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-21T10:21:13.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-21T10:21:13.891 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-21T10:21:13.892 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-21T10:21:13.892 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-21T10:21:13.892 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-21T10:21:13.892 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-21T10:21:13.892 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-21T10:21:13.892 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-21T10:21:13.892 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.arena_block_size: 1048576 2024-04-21T10:21:13.892 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-21T10:21:13.892 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-21T10:21:13.892 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.disable_auto_compactions: 0 2024-04-21T10:21:13.893 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-21T10:21:13.893 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-21T10:21:13.893 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-21T10:21:13.893 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-21T10:21:13.893 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-21T10:21:13.893 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-21T10:21:13.893 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-21T10:21:13.893 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-21T10:21:13.893 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-21T10:21:13.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-21T10:21:13.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-21T10:21:13.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.inplace_update_support: 0 2024-04-21T10:21:13.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-21T10:21:13.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-21T10:21:13.894 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-21T10:21:13.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-21T10:21:13.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.bloom_locality: 0 2024-04-21T10:21:13.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.max_successive_merges: 0 2024-04-21T10:21:13.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-21T10:21:13.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.paranoid_file_checks: 0 2024-04-21T10:21:13.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.force_consistency_checks: 1 2024-04-21T10:21:13.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.report_bg_io_stats: 0 2024-04-21T10:21:13.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.ttl: 2592000 2024-04-21T10:21:13.895 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-21T10:21:13.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-21T10:21:13.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-21T10:21:13.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.enable_blob_files: false 2024-04-21T10:21:13.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.min_blob_size: 0 2024-04-21T10:21:13.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.blob_file_size: 268435456 2024-04-21T10:21:13.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-21T10:21:13.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-21T10:21:13.896 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-21T10:21:13.897 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-21T10:21:13.897 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-21T10:21:13.897 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.blob_file_starting_level: 0 2024-04-21T10:21:13.897 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-21T10:21:13.897 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-21T10:21:13.897 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-21T10:21:13.897 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1e6a8579-7b34-4dd2-a45f-871a7d8ab881 2024-04-21T10:21:13.897 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713694873702562, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-21T10:21:13.897 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-21T10:21:13.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713694873703587, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101076, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98965, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14405, "raw_average_key_size": 49, "raw_value_size": 91144, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713694873, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1e6a8579-7b34-4dd2-a45f-871a7d8ab881", "db_session_id": "S27ZS24ZV5LOBJSER52N", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-21T10:21:13.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713694873703671, "job": 1, "event": "recovery_finished"} 2024-04-21T10:21:13.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-21T10:21:13.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-21T10:21:13.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5648ad514000 2024-04-21T10:21:13.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: DB pointer 0x5648ad4fe000 2024-04-21T10:21:13.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-21T10:21:13.898 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-21T10:21:13.899 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** DB Stats ** 2024-04-21T10:21:13.899 INFO:journalctl@ceph.mon.a.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T10:21:13.899 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-21T10:21:13.899 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T10:21:13.899 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T10:21:13.899 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-21T10:21:13.899 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T10:21:13.899 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T10:21:13.900 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-04-21T10:21:13.900 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** Compaction Stats [default] ** 2024-04-21T10:21:13.900 INFO:journalctl@ceph.mon.a.smithi026.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T10:21:13.900 INFO:journalctl@ceph.mon.a.smithi026.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-21T10:21:13.900 INFO:journalctl@ceph.mon.a.smithi026.stdout: L0 2/0 100.57 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T10:21:13.900 INFO:journalctl@ceph.mon.a.smithi026.stdout: Sum 2/0 100.57 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T10:21:13.900 INFO:journalctl@ceph.mon.a.smithi026.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T10:21:13.900 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-04-21T10:21:13.900 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** Compaction Stats [default] ** 2024-04-21T10:21:13.901 INFO:journalctl@ceph.mon.a.smithi026.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T10:21:13.901 INFO:journalctl@ceph.mon.a.smithi026.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-21T10:21:13.901 INFO:journalctl@ceph.mon.a.smithi026.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 157.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T10:21:13.901 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-04-21T10:21:13.901 INFO:journalctl@ceph.mon.a.smithi026.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-21T10:21:13.901 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-04-21T10:21:13.901 INFO:journalctl@ceph.mon.a.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T10:21:13.901 INFO:journalctl@ceph.mon.a.smithi026.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-21T10:21:13.901 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative compaction: 0.00 GB write, 30.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval compaction: 0.00 GB write, 30.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout: Block cache BinnedLRUCache@0x5648ad3ff1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: starting mon.a rank 0 at public addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] at bind addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: mon.a@-1(???) e1 preinit fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:21:13.902 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: mon.a@-1(???).mds e1 new map 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: mon.a@-1(???).mds e1 print_map 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout: e1 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout: legacy client fscid: -1 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout: No filesystems configured 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 bash[27946]: 1526264d8981cbd07a9d45e7c0d71bc634e1809f178ac78a012ae3044bff604f 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 systemd[1]: Started Ceph mon.a for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:21:13.903 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-21T10:21:13.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: monmap epoch 1 2024-04-21T10:21:13.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:21:13.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: last_changed 2024-04-21T10:21:05.761117+0000 2024-04-21T10:21:13.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: created 2024-04-21T10:21:05.761117+0000 2024-04-21T10:21:13.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: min_mon_release 19 (squid) 2024-04-21T10:21:13.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: election_strategy: 1 2024-04-21T10:21:13.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-04-21T10:21:13.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: fsmap 2024-04-21T10:21:13.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: osdmap e1: 0 total, 0 up, 0 in 2024-04-21T10:21:13.904 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:13 smithi026 ceph-mon[27965]: mgrmap e1: no daemons active 2024-04-21T10:21:14.873 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.244+0000 7f2c11de0700 1 Processor -- start 2024-04-21T10:21:14.873 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.244+0000 7f2c11de0700 1 -- start start 2024-04-21T10:21:14.873 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.244+0000 7f2c11de0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 0x7f2c0c108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:14.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.244+0000 7f2c11de0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c109060 con 0x7f2c0c108670 2024-04-21T10:21:14.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.244+0000 7f2c0b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 0x7f2c0c108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:14.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.245+0000 7f2c0b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 0x7f2c0c108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45722/0 (socket says 172.21.15.26:45722) 2024-04-21T10:21:14.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.245+0000 7f2c0b7fe700 1 -- 172.21.15.26:0/2928646595 learned_addr learned my addr 172.21.15.26:0/2928646595 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:14.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.245+0000 7f2c0b7fe700 1 -- 172.21.15.26:0/2928646595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c109820 con 0x7f2c0c108670 2024-04-21T10:21:14.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.245+0000 7f2c0b7fe700 1 --2- 172.21.15.26:0/2928646595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 0x7f2c0c108a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009e80 tx=0x7f2c0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d10998a8578d99 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:14.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.245+0000 7f2c0a7fc700 1 -- 172.21.15.26:0/2928646595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c04017070 con 0x7f2c0c108670 2024-04-21T10:21:14.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.245+0000 7f2c0a7fc700 1 -- 172.21.15.26:0/2928646595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2c0400b8c0 con 0x7f2c0c108670 2024-04-21T10:21:14.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.245+0000 7f2c0a7fc700 1 -- 172.21.15.26:0/2928646595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c04004490 con 0x7f2c0c108670 2024-04-21T10:21:14.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.246+0000 7f2c11de0700 1 -- 172.21.15.26:0/2928646595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 msgr2=0x7f2c0c108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:14.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.246+0000 7f2c11de0700 1 --2- 172.21.15.26:0/2928646595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 0x7f2c0c108a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009e80 tx=0x7f2c0400b330 comp rx=0 tx=0).stop 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.246+0000 7f2c11de0700 1 -- 172.21.15.26:0/2928646595 shutdown_connections 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.246+0000 7f2c11de0700 1 --2- 172.21.15.26:0/2928646595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 0x7f2c0c108a90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.246+0000 7f2c11de0700 1 -- 172.21.15.26:0/2928646595 >> 172.21.15.26:0/2928646595 conn(0x7f2c0c103c30 msgr2=0x7f2c0c106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.246+0000 7f2c11de0700 1 -- 172.21.15.26:0/2928646595 shutdown_connections 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.246+0000 7f2c11de0700 1 -- 172.21.15.26:0/2928646595 wait complete. 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.246+0000 7f2c11de0700 1 Processor -- start 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.247+0000 7f2c11de0700 1 -- start start 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.247+0000 7f2c11de0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 0x7f2c0c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.247+0000 7f2c11de0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10ca50 con 0x7f2c0c108670 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.247+0000 7f2c0b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 0x7f2c0c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.247+0000 7f2c0b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 0x7f2c0c19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45732/0 (socket says 172.21.15.26:45732) 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.247+0000 7f2c0b7fe700 1 -- 172.21.15.26:0/1596752933 learned_addr learned my addr 172.21.15.26:0/1596752933 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.247+0000 7f2c0b7fe700 1 -- 172.21.15.26:0/1596752933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c19ca30 con 0x7f2c0c108670 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.247+0000 7f2c0b7fe700 1 --2- 172.21.15.26:0/1596752933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 0x7f2c0c19c4f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2c04004a60 tx=0x7f2c0400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:14.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.248+0000 7f2c08ff9700 1 -- 172.21.15.26:0/1596752933 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c04020020 con 0x7f2c0c108670 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.248+0000 7f2c08ff9700 1 -- 172.21.15.26:0/1596752933 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2c04012430 con 0x7f2c0c108670 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.248+0000 7f2c11de0700 1 -- 172.21.15.26:0/1596752933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0c19ccc0 con 0x7f2c0c108670 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.248+0000 7f2c08ff9700 1 -- 172.21.15.26:0/1596752933 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c04017040 con 0x7f2c0c108670 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.248+0000 7f2c11de0700 1 -- 172.21.15.26:0/1596752933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0c19f9b0 con 0x7f2c0c108670 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.248+0000 7f2c08ff9700 1 -- 172.21.15.26:0/1596752933 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2c04023020 con 0x7f2c0c108670 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.248+0000 7f2c08ff9700 1 -- 172.21.15.26:0/1596752933 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2c0400f050 con 0x7f2c0c108670 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.249+0000 7f2c11de0700 1 -- 172.21.15.26:0/1596752933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0c10cbd0 con 0x7f2c0c108670 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.251+0000 7f2c08ff9700 1 -- 172.21.15.26:0/1596752933 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2c0402a020 con 0x7f2c0c108670 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.322+0000 7f2c11de0700 1 -- 172.21.15.26:0/1596752933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f2c0c19fbf0 con 0x7f2c0c108670 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.323+0000 7f2c08ff9700 1 -- 172.21.15.26:0/1596752933 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f2c0400b860 con 0x7f2c0c108670 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.324+0000 7f2c11de0700 1 -- 172.21.15.26:0/1596752933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 msgr2=0x7f2c0c19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.324+0000 7f2c11de0700 1 --2- 172.21.15.26:0/1596752933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 0x7f2c0c19c4f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2c04004a60 tx=0x7f2c0400bd50 comp rx=0 tx=0).stop 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.324+0000 7f2c11de0700 1 -- 172.21.15.26:0/1596752933 shutdown_connections 2024-04-21T10:21:14.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.324+0000 7f2c11de0700 1 --2- 172.21.15.26:0/1596752933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c108670 0x7f2c0c19c4f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:14.877 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.324+0000 7f2c11de0700 1 -- 172.21.15.26:0/1596752933 >> 172.21.15.26:0/1596752933 conn(0x7f2c0c103c30 msgr2=0x7f2c0c104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:14.877 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.324+0000 7f2c11de0700 1 -- 172.21.15.26:0/1596752933 shutdown_connections 2024-04-21T10:21:14.877 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:14.324+0000 7f2c11de0700 1 -- 172.21.15.26:0/1596752933 wait complete. 2024-04-21T10:21:14.877 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-21T10:21:14.900 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-21T10:21:14.900 INFO:teuthology.orchestra.run.smithi026.stdout:Creating mgr... 2024-04-21T10:21:14.900 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-21T10:21:14.901 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-21T10:21:15.142 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.x 2024-04-21T10:21:15.142 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.x.service: Unit ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-21T10:21:15.305 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b77245c8-ffc8-11ee-bc93-c7b262605968.target.wants/ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@.service. 2024-04-21T10:21:15.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:15 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1596752933' entity='client.admin' 2024-04-21T10:21:15.793 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T10:21:15.794 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-04-21T10:21:15.801 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T10:21:15.801 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-04-21T10:21:15.801 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-04-21T10:21:15.801 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to enable service . firewalld.service is not available 2024-04-21T10:21:15.809 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T10:21:15.809 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-04-21T10:21:15.817 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T10:21:15.817 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-04-21T10:21:15.817 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-04-21T10:21:15.817 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-21T10:21:15.817 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr to start... 2024-04-21T10:21:15.817 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr... 2024-04-21T10:21:16.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:16 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2877848349' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T10:21:16.685 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:16 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:16.614+0000 7f9bf0efa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T10:21:16.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:16.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-04-21T10:21:16.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:21:16.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T10:21:16.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T10:21:16.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T10:21:16.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T10:21:16.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:16.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T10:21:16.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T10:21:16.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T10:21:16.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T10:21:16.984 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T10:21:16.984 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T10:21:16.984 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T10:21:16.984 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:16.984 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T10:21:16.984 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T10:21:16.985 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T10:21:08.638142+0000", 2024-04-21T10:21:16.986 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.293+0000 7f78378c6700 1 Processor -- start 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.293+0000 7f78378c6700 1 -- start start 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.293+0000 7f78378c6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 0x7f783010b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.293+0000 7f78378c6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f783007cdd0 con 0x7f7830109160 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.294+0000 7f7835662700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 0x7f783010b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.294+0000 7f7835662700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 0x7f783010b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34758/0 (socket says 172.21.15.26:34758) 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.294+0000 7f7835662700 1 -- 172.21.15.26:0/3347639391 learned_addr learned my addr 172.21.15.26:0/3347639391 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.294+0000 7f7835662700 1 -- 172.21.15.26:0/3347639391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783010ba90 con 0x7f7830109160 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.294+0000 7f7835662700 1 --2- 172.21.15.26:0/3347639391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 0x7f783010b550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f782c0098a0 tx=0x7f782c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd42f171f700816 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.295+0000 7f7827fff700 1 -- 172.21.15.26:0/3347639391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f782c004160 con 0x7f7830109160 2024-04-21T10:21:16.987 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.295+0000 7f7827fff700 1 -- 172.21.15.26:0/3347639391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f782c004300 con 0x7f7830109160 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.295+0000 7f7827fff700 1 -- 172.21.15.26:0/3347639391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f782c011af0 con 0x7f7830109160 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.295+0000 7f78378c6700 1 -- 172.21.15.26:0/3347639391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 msgr2=0x7f783010b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.295+0000 7f78378c6700 1 --2- 172.21.15.26:0/3347639391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 0x7f783010b550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f782c0098a0 tx=0x7f782c009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.295+0000 7f78378c6700 1 -- 172.21.15.26:0/3347639391 shutdown_connections 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.295+0000 7f78378c6700 1 --2- 172.21.15.26:0/3347639391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 0x7f783010b550 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.295+0000 7f78378c6700 1 -- 172.21.15.26:0/3347639391 >> 172.21.15.26:0/3347639391 conn(0x7f7830102b10 msgr2=0x7f7830104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.296+0000 7f78378c6700 1 -- 172.21.15.26:0/3347639391 shutdown_connections 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.296+0000 7f78378c6700 1 -- 172.21.15.26:0/3347639391 wait complete. 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.296+0000 7f78378c6700 1 Processor -- start 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.296+0000 7f78378c6700 1 -- start start 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.296+0000 7f78378c6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 0x7f783019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.296+0000 7f78378c6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f783010c920 con 0x7f7830109160 2024-04-21T10:21:16.988 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.297+0000 7f7835662700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 0x7f783019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:16.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.297+0000 7f7835662700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 0x7f783019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34766/0 (socket says 172.21.15.26:34766) 2024-04-21T10:21:16.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.297+0000 7f7835662700 1 -- 172.21.15.26:0/2877848349 learned_addr learned my addr 172.21.15.26:0/2877848349 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:16.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.297+0000 7f7835662700 1 -- 172.21.15.26:0/2877848349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783019ca40 con 0x7f7830109160 2024-04-21T10:21:16.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.297+0000 7f7835662700 1 --2- 172.21.15.26:0/2877848349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 0x7f783019c500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f782c009fb0 tx=0x7f782c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:16.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.297+0000 7f78267fc700 1 -- 172.21.15.26:0/2877848349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f782c0036a0 con 0x7f7830109160 2024-04-21T10:21:16.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.297+0000 7f78267fc700 1 -- 172.21.15.26:0/2877848349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f782c01a430 con 0x7f7830109160 2024-04-21T10:21:16.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.297+0000 7f78378c6700 1 -- 172.21.15.26:0/2877848349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783019ccd0 con 0x7f7830109160 2024-04-21T10:21:16.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.297+0000 7f78267fc700 1 -- 172.21.15.26:0/2877848349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f782c019550 con 0x7f7830109160 2024-04-21T10:21:16.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.297+0000 7f78378c6700 1 -- 172.21.15.26:0/2877848349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783019f9c0 con 0x7f7830109160 2024-04-21T10:21:16.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.298+0000 7f78267fc700 1 -- 172.21.15.26:0/2877848349 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f782c019be0 con 0x7f7830109160 2024-04-21T10:21:16.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.298+0000 7f78267fc700 1 -- 172.21.15.26:0/2877848349 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f782c024730 con 0x7f7830109160 2024-04-21T10:21:16.989 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.299+0000 7f78378c6700 1 -- 172.21.15.26:0/2877848349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78300640a0 con 0x7f7830109160 2024-04-21T10:21:16.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.301+0000 7f78267fc700 1 -- 172.21.15.26:0/2877848349 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f782c043e50 con 0x7f7830109160 2024-04-21T10:21:16.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.394+0000 7f78378c6700 1 -- 172.21.15.26:0/2877848349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7830107320 con 0x7f7830109160 2024-04-21T10:21:16.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.394+0000 7f78267fc700 1 -- 172.21.15.26:0/2877848349 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f782c032030 con 0x7f7830109160 2024-04-21T10:21:16.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.395+0000 7f78378c6700 1 -- 172.21.15.26:0/2877848349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 msgr2=0x7f783019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:16.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.395+0000 7f78378c6700 1 --2- 172.21.15.26:0/2877848349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 0x7f783019c500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f782c009fb0 tx=0x7f782c011940 comp rx=0 tx=0).stop 2024-04-21T10:21:16.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.395+0000 7f78378c6700 1 -- 172.21.15.26:0/2877848349 shutdown_connections 2024-04-21T10:21:16.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.395+0000 7f78378c6700 1 --2- 172.21.15.26:0/2877848349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7830109160 0x7f783019c500 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:16.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.395+0000 7f78378c6700 1 -- 172.21.15.26:0/2877848349 >> 172.21.15.26:0/2877848349 conn(0x7f7830102b10 msgr2=0x7f7830103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:16.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.395+0000 7f78378c6700 1 -- 172.21.15.26:0/2877848349 shutdown_connections 2024-04-21T10:21:16.990 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:16.395+0000 7f78378c6700 1 -- 172.21.15.26:0/2877848349 wait complete. 2024-04-21T10:21:16.990 INFO:teuthology.orchestra.run.smithi026.stdout:mgr not available, waiting (1/15)... 2024-04-21T10:21:18.186 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:17 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:17.871+0000 7f9bf0efa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T10:21:18.186 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:18 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:18.047+0000 7f9bf0efa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T10:21:18.523 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:18 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:18.223+0000 7f9bf0efa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T10:21:18.523 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:18 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:18.523+0000 7f9bf0efa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T10:21:18.828 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:18 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:18.618+0000 7f9bf0efa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T10:21:18.828 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:18 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:18.828+0000 7f9bf0efa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T10:21:19.102 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:19 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:19.032+0000 7f9bf0efa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:21:19.672 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:19 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1866974195' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T10:21:19.935 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:19 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:19.672+0000 7f9bf0efa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T10:21:20.070 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T10:21:20.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-04-21T10:21:20.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-21T10:21:20.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T10:21:20.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:20.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T10:21:20.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T10:21:20.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:20.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T10:21:20.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:20.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T10:21:20.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T10:21:20.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T10:21:20.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T10:21:20.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T10:21:20.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T10:21:20.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:20.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T10:21:20.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T10:21:20.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T10:21:20.080 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T10:21:20.081 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T10:21:08.638142+0000", 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-04-21T10:21:20.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.372+0000 7fd2ca144700 1 Processor -- start 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.372+0000 7fd2ca144700 1 -- start start 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.373+0000 7fd2ca144700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 0x7fd2c410b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.373+0000 7fd2ca144700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c407ce70 con 0x7fd2c41090c0 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.373+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 0x7fd2c410b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.373+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 0x7fd2c410b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34774/0 (socket says 172.21.15.26:34774) 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.373+0000 7fd2c37fe700 1 -- 172.21.15.26:0/1635079770 learned_addr learned my addr 172.21.15.26:0/1635079770 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.373+0000 7fd2c37fe700 1 -- 172.21.15.26:0/1635079770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c410b9f0 con 0x7fd2c41090c0 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.374+0000 7fd2c37fe700 1 --2- 172.21.15.26:0/1635079770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 0x7fd2c410b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40098a0 tx=0x7fd2b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f801c02615d8cd83 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.374+0000 7fd2c27fc700 1 -- 172.21.15.26:0/1635079770 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2b4004160 con 0x7fd2c41090c0 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.374+0000 7fd2c27fc700 1 -- 172.21.15.26:0/1635079770 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd2b4004300 con 0x7fd2c41090c0 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.374+0000 7fd2c27fc700 1 -- 172.21.15.26:0/1635079770 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2b4011af0 con 0x7fd2c41090c0 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.375+0000 7fd2ca144700 1 -- 172.21.15.26:0/1635079770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 msgr2=0x7fd2c410b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.375+0000 7fd2ca144700 1 --2- 172.21.15.26:0/1635079770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 0x7fd2c410b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40098a0 tx=0x7fd2b4009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:20.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.375+0000 7fd2ca144700 1 -- 172.21.15.26:0/1635079770 shutdown_connections 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.375+0000 7fd2ca144700 1 --2- 172.21.15.26:0/1635079770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 0x7fd2c410b4b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.375+0000 7fd2ca144700 1 -- 172.21.15.26:0/1635079770 >> 172.21.15.26:0/1635079770 conn(0x7fd2c4102b10 msgr2=0x7fd2c4104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.375+0000 7fd2ca144700 1 -- 172.21.15.26:0/1635079770 shutdown_connections 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.375+0000 7fd2ca144700 1 -- 172.21.15.26:0/1635079770 wait complete. 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.375+0000 7fd2ca144700 1 Processor -- start 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.376+0000 7fd2ca144700 1 -- start start 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.376+0000 7fd2ca144700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 0x7fd2c4108940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.376+0000 7fd2ca144700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c410c940 con 0x7fd2c41090c0 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.376+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 0x7fd2c4108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.376+0000 7fd2c37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 0x7fd2c4108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34784/0 (socket says 172.21.15.26:34784) 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.376+0000 7fd2c37fe700 1 -- 172.21.15.26:0/1866974195 learned_addr learned my addr 172.21.15.26:0/1866974195 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.376+0000 7fd2c37fe700 1 -- 172.21.15.26:0/1866974195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c4108e80 con 0x7fd2c41090c0 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.376+0000 7fd2c37fe700 1 --2- 172.21.15.26:0/1866974195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 0x7fd2c4108940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009fb0 tx=0x7fd2b4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.377+0000 7fd2c0ff9700 1 -- 172.21.15.26:0/1866974195 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2b40036a0 con 0x7fd2c41090c0 2024-04-21T10:21:20.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.377+0000 7fd2ca144700 1 -- 172.21.15.26:0/1866974195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c4107070 con 0x7fd2c41090c0 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.377+0000 7fd2c0ff9700 1 -- 172.21.15.26:0/1866974195 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd2b401a430 con 0x7fd2c41090c0 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.377+0000 7fd2c0ff9700 1 -- 172.21.15.26:0/1866974195 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2b4019550 con 0x7fd2c41090c0 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.377+0000 7fd2ca144700 1 -- 172.21.15.26:0/1866974195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c41075b0 con 0x7fd2c41090c0 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.377+0000 7fd2c0ff9700 1 -- 172.21.15.26:0/1866974195 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd2b4019be0 con 0x7fd2c41090c0 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.377+0000 7fd2c0ff9700 1 -- 172.21.15.26:0/1866974195 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd2b4024730 con 0x7fd2c41090c0 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.378+0000 7fd2ca144700 1 -- 172.21.15.26:0/1866974195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c40640a0 con 0x7fd2c41090c0 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.380+0000 7fd2c0ff9700 1 -- 172.21.15.26:0/1866974195 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd2b4043e50 con 0x7fd2c41090c0 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.454+0000 7fd2ca144700 1 -- 172.21.15.26:0/1866974195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd2c41078f0 con 0x7fd2c41090c0 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.454+0000 7fd2c0ff9700 1 -- 172.21.15.26:0/1866974195 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd2b4032030 con 0x7fd2c41090c0 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.456+0000 7fd2ca144700 1 -- 172.21.15.26:0/1866974195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 msgr2=0x7fd2c4108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.456+0000 7fd2ca144700 1 --2- 172.21.15.26:0/1866974195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 0x7fd2c4108940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009fb0 tx=0x7fd2b4011940 comp rx=0 tx=0).stop 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.456+0000 7fd2ca144700 1 -- 172.21.15.26:0/1866974195 shutdown_connections 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.456+0000 7fd2ca144700 1 --2- 172.21.15.26:0/1866974195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2c41090c0 0x7fd2c4108940 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.456+0000 7fd2ca144700 1 -- 172.21.15.26:0/1866974195 >> 172.21.15.26:0/1866974195 conn(0x7fd2c4102b10 msgr2=0x7fd2c4103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:20.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.456+0000 7fd2ca144700 1 -- 172.21.15.26:0/1866974195 shutdown_connections 2024-04-21T10:21:20.086 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:19.456+0000 7fd2ca144700 1 -- 172.21.15.26:0/1866974195 wait complete. 2024-04-21T10:21:20.086 INFO:teuthology.orchestra.run.smithi026.stdout:mgr not available, waiting (2/15)... 2024-04-21T10:21:20.418 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:20 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:20.110+0000 7f9bf0efa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T10:21:20.419 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:20 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:20.209+0000 7f9bf0efa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T10:21:20.419 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:20 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:20.418+0000 7f9bf0efa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T10:21:21.307 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:20 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:20.961+0000 7f9bf0efa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T10:21:21.308 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:21 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:21.056+0000 7f9bf0efa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T10:21:21.661 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:21 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:21.307+0000 7f9bf0efa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T10:21:21.661 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:21 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:21.445+0000 7f9bf0efa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T10:21:21.661 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:21 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:21.542+0000 7f9bf0efa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T10:21:21.661 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:21 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:21.661+0000 7f9bf0efa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T10:21:22.290 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:21 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:21.973+0000 7f9bf0efa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T10:21:22.837 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:22 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/614884451' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T10:21:22.838 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:22 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:22.613+0000 7f9bf0efa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T10:21:22.838 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:22 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:22.837+0000 7f9bf0efa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T10:21:23.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:23.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-04-21T10:21:23.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:21:23.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T10:21:23.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T10:21:23.144 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T10:21:23.145 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:23.146 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T10:21:23.146 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:23.146 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T10:21:23.146 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T10:21:23.146 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T10:21:23.154 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T10:21:23.155 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T10:21:08.638142+0000", 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.460+0000 7fc3207a2700 1 Processor -- start 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.460+0000 7fc3207a2700 1 -- start start 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.461+0000 7fc3207a2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 0x7fc318108a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.461+0000 7fc3207a2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc318109050 con 0x7fc318108660 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.461+0000 7fc31e53e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 0x7fc318108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.461+0000 7fc31e53e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 0x7fc318108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34790/0 (socket says 172.21.15.26:34790) 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.461+0000 7fc31e53e700 1 -- 172.21.15.26:0/243949043 learned_addr learned my addr 172.21.15.26:0/243949043 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.461+0000 7fc31e53e700 1 -- 172.21.15.26:0/243949043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc318109810 con 0x7fc318108660 2024-04-21T10:21:23.157 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.461+0000 7fc31e53e700 1 --2- 172.21.15.26:0/243949043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 0x7fc318108a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3080098a0 tx=0x7fc308009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1336238845318e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.461+0000 7fc31d53c700 1 -- 172.21.15.26:0/243949043 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc308004160 con 0x7fc318108660 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.462+0000 7fc31d53c700 1 -- 172.21.15.26:0/243949043 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc308004300 con 0x7fc318108660 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.462+0000 7fc31d53c700 1 -- 172.21.15.26:0/243949043 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc308011af0 con 0x7fc318108660 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.462+0000 7fc3207a2700 1 -- 172.21.15.26:0/243949043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 msgr2=0x7fc318108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.462+0000 7fc3207a2700 1 --2- 172.21.15.26:0/243949043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 0x7fc318108a80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3080098a0 tx=0x7fc308009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.462+0000 7fc3207a2700 1 -- 172.21.15.26:0/243949043 shutdown_connections 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.462+0000 7fc3207a2700 1 --2- 172.21.15.26:0/243949043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 0x7fc318108a80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.462+0000 7fc3207a2700 1 -- 172.21.15.26:0/243949043 >> 172.21.15.26:0/243949043 conn(0x7fc318103c40 msgr2=0x7fc318106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.462+0000 7fc3207a2700 1 -- 172.21.15.26:0/243949043 shutdown_connections 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.462+0000 7fc3207a2700 1 -- 172.21.15.26:0/243949043 wait complete. 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.463+0000 7fc3207a2700 1 Processor -- start 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.463+0000 7fc3207a2700 1 -- start start 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.463+0000 7fc3207a2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 0x7fc31807eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:23.158 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.463+0000 7fc3207a2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc31810ca40 con 0x7fc318108660 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.463+0000 7fc31e53e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 0x7fc31807eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.463+0000 7fc31e53e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 0x7fc31807eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34802/0 (socket says 172.21.15.26:34802) 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.463+0000 7fc31e53e700 1 -- 172.21.15.26:0/614884451 learned_addr learned my addr 172.21.15.26:0/614884451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.464+0000 7fc31e53e700 1 -- 172.21.15.26:0/614884451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31807f0b0 con 0x7fc318108660 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.464+0000 7fc31e53e700 1 --2- 172.21.15.26:0/614884451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 0x7fc31807eb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc3080117b0 tx=0x7fc308011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.464+0000 7fc30f7fe700 1 -- 172.21.15.26:0/614884451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc308003af0 con 0x7fc318108660 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.464+0000 7fc3207a2700 1 -- 172.21.15.26:0/614884451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31807d2a0 con 0x7fc318108660 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.464+0000 7fc30f7fe700 1 -- 172.21.15.26:0/614884451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc308020070 con 0x7fc318108660 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.464+0000 7fc30f7fe700 1 -- 172.21.15.26:0/614884451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc308019630 con 0x7fc318108660 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.464+0000 7fc3207a2700 1 -- 172.21.15.26:0/614884451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc31807d7e0 con 0x7fc318108660 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.464+0000 7fc30f7fe700 1 -- 172.21.15.26:0/614884451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc308019c80 con 0x7fc318108660 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.464+0000 7fc30f7fe700 1 -- 172.21.15.26:0/614884451 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc30800b720 con 0x7fc318108660 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.465+0000 7fc3207a2700 1 -- 172.21.15.26:0/614884451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3180640a0 con 0x7fc318108660 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.467+0000 7fc30f7fe700 1 -- 172.21.15.26:0/614884451 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc3080042c0 con 0x7fc318108660 2024-04-21T10:21:23.159 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.541+0000 7fc3207a2700 1 -- 172.21.15.26:0/614884451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc31807dd10 con 0x7fc318108660 2024-04-21T10:21:23.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.542+0000 7fc30f7fe700 1 -- 172.21.15.26:0/614884451 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc30801a9c0 con 0x7fc318108660 2024-04-21T10:21:23.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.543+0000 7fc3207a2700 1 -- 172.21.15.26:0/614884451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 msgr2=0x7fc31807eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:23.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.543+0000 7fc3207a2700 1 --2- 172.21.15.26:0/614884451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 0x7fc31807eb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc3080117b0 tx=0x7fc308011b70 comp rx=0 tx=0).stop 2024-04-21T10:21:23.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.543+0000 7fc3207a2700 1 -- 172.21.15.26:0/614884451 shutdown_connections 2024-04-21T10:21:23.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.543+0000 7fc3207a2700 1 --2- 172.21.15.26:0/614884451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc318108660 0x7fc31807eb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:23.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.543+0000 7fc3207a2700 1 -- 172.21.15.26:0/614884451 >> 172.21.15.26:0/614884451 conn(0x7fc318103c40 msgr2=0x7fc318104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:23.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.543+0000 7fc3207a2700 1 -- 172.21.15.26:0/614884451 shutdown_connections 2024-04-21T10:21:23.160 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:22.543+0000 7fc3207a2700 1 -- 172.21.15.26:0/614884451 wait complete. 2024-04-21T10:21:23.160 INFO:teuthology.orchestra.run.smithi026.stdout:mgr not available, waiting (3/15)... 2024-04-21T10:21:23.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: Activating manager daemon x 2024-04-21T10:21:23.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: mgrmap e2: x(active, starting, since 0.00312238s) 2024-04-21T10:21:23.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: from='mgr.14100 172.21.15.26:0/3907872444' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T10:21:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: from='mgr.14100 172.21.15.26:0/3907872444' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T10:21:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: from='mgr.14100 172.21.15.26:0/3907872444' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T10:21:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: from='mgr.14100 172.21.15.26:0/3907872444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:21:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: from='mgr.14100 172.21.15.26:0/3907872444' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T10:21:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: Manager daemon x is now available 2024-04-21T10:21:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: from='mgr.14100 172.21.15.26:0/3907872444' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T10:21:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: from='mgr.14100 172.21.15.26:0/3907872444' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T10:21:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: from='mgr.14100 172.21.15.26:0/3907872444' entity='mgr.x' 2024-04-21T10:21:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: from='mgr.14100 172.21.15.26:0/3907872444' entity='mgr.x' 2024-04-21T10:21:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:23 smithi026 ceph-mon[27965]: from='mgr.14100 172.21.15.26:0/3907872444' entity='mgr.x' 2024-04-21T10:21:25.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:24 smithi026 ceph-mon[27965]: mgrmap e3: x(active, since 1.00716s) 2024-04-21T10:21:26.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:25 smithi026 ceph-mon[27965]: mgrmap e4: x(active, since 2s) 2024-04-21T10:21:26.449 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-04-21T10:21:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:21:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T10:21:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T10:21:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T10:21:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T10:21:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T10:21:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T10:21:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2024-04-21T10:21:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-04-21T10:21:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T10:21:26.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T10:21:26.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-04-21T10:21:26.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-21T10:21:26.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T10:21:26.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:26.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T10:21:26.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T10:21:26.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:26.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T10:21:26.451 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:26.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T10:21:26.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T10:21:26.452 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T10:21:26.462 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T10:21:26.462 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T10:21:26.462 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T10:21:26.462 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:26.462 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T10:21:26.462 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T10:21:26.462 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T10:21:26.462 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T10:21:26.463 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T10:21:26.463 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T10:21:26.463 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T10:21:26.463 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T10:21:26.463 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T10:21:26.463 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:26.463 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T10:21:26.463 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:26.463 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T10:21:26.463 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T10:21:26.463 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:26.463 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T10:21:08.638142+0000", 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T10:21:26.464 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2024-04-21T10:21:26.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T10:21:26.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-04-21T10:21:26.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.569+0000 7f077f410700 1 Processor -- start 2024-04-21T10:21:26.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.569+0000 7f077f410700 1 -- start start 2024-04-21T10:21:26.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.570+0000 7f077f410700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 0x7f077810b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:26.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.570+0000 7f077f410700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0778063d60 con 0x7f07781090b0 2024-04-21T10:21:26.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.570+0000 7f077d1ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 0x7f077810b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:26.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.570+0000 7f077d1ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 0x7f077810b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53002/0 (socket says 172.21.15.26:53002) 2024-04-21T10:21:26.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.570+0000 7f077d1ac700 1 -- 172.21.15.26:0/625569489 learned_addr learned my addr 172.21.15.26:0/625569489 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:26.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.570+0000 7f077d1ac700 1 -- 172.21.15.26:0/625569489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077810b9e0 con 0x7f07781090b0 2024-04-21T10:21:26.465 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.570+0000 7f077d1ac700 1 --2- 172.21.15.26:0/625569489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 0x7f077810b4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f07740098a0 tx=0x7f0774009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19183abea711139b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.570+0000 7f076ffff700 1 -- 172.21.15.26:0/625569489 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0774004160 con 0x7f07781090b0 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.571+0000 7f076ffff700 1 -- 172.21.15.26:0/625569489 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0774004300 con 0x7f07781090b0 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.571+0000 7f076ffff700 1 -- 172.21.15.26:0/625569489 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0774011af0 con 0x7f07781090b0 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.571+0000 7f077f410700 1 -- 172.21.15.26:0/625569489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 msgr2=0x7f077810b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.571+0000 7f077f410700 1 --2- 172.21.15.26:0/625569489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 0x7f077810b4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f07740098a0 tx=0x7f0774009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.571+0000 7f077f410700 1 -- 172.21.15.26:0/625569489 shutdown_connections 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.571+0000 7f077f410700 1 --2- 172.21.15.26:0/625569489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 0x7f077810b4a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.571+0000 7f077f410700 1 -- 172.21.15.26:0/625569489 >> 172.21.15.26:0/625569489 conn(0x7f0778102b80 msgr2=0x7f0778104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.571+0000 7f077f410700 1 -- 172.21.15.26:0/625569489 shutdown_connections 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.571+0000 7f077f410700 1 -- 172.21.15.26:0/625569489 wait complete. 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.572+0000 7f077f410700 1 Processor -- start 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.572+0000 7f077f410700 1 -- start start 2024-04-21T10:21:26.466 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.572+0000 7f077f410700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 0x7f077819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:26.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.572+0000 7f077f410700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f077810c940 con 0x7f07781090b0 2024-04-21T10:21:26.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.572+0000 7f077d1ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 0x7f077819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:26.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.572+0000 7f077d1ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 0x7f077819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53018/0 (socket says 172.21.15.26:53018) 2024-04-21T10:21:26.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.572+0000 7f077d1ac700 1 -- 172.21.15.26:0/1626923656 learned_addr learned my addr 172.21.15.26:0/1626923656 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:26.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.573+0000 7f077d1ac700 1 -- 172.21.15.26:0/1626923656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077819ec90 con 0x7f07781090b0 2024-04-21T10:21:26.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.573+0000 7f077d1ac700 1 --2- 172.21.15.26:0/1626923656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 0x7f077819e750 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0774009fb0 tx=0x7f0774011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:26.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.573+0000 7f076e7fc700 1 -- 172.21.15.26:0/1626923656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07740036a0 con 0x7f07781090b0 2024-04-21T10:21:26.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.573+0000 7f076e7fc700 1 -- 172.21.15.26:0/1626923656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f077401a430 con 0x7f07781090b0 2024-04-21T10:21:26.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.573+0000 7f076e7fc700 1 -- 172.21.15.26:0/1626923656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0774019550 con 0x7f07781090b0 2024-04-21T10:21:26.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.573+0000 7f077f410700 1 -- 172.21.15.26:0/1626923656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077819ef20 con 0x7f07781090b0 2024-04-21T10:21:26.467 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.573+0000 7f077f410700 1 -- 172.21.15.26:0/1626923656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07781a1c10 con 0x7f07781090b0 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.574+0000 7f076e7fc700 1 -- 172.21.15.26:0/1626923656 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f0774003840 con 0x7f07781090b0 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.574+0000 7f076e7fc700 1 --2- 172.21.15.26:0/1626923656 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0764039510 0x7f076403b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.574+0000 7f077f410700 1 -- 172.21.15.26:0/1626923656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07780640a0 con 0x7f07781090b0 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.574+0000 7f076e7fc700 1 -- 172.21.15.26:0/1626923656 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f077404d8a0 con 0x7f07781090b0 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.574+0000 7f077c9ab700 1 --2- 172.21.15.26:0/1626923656 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0764039510 0x7f076403b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.575+0000 7f077c9ab700 1 --2- 172.21.15.26:0/1626923656 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0764039510 0x7f076403b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0768006fd0 tx=0x7f0768006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.580+0000 7f076e7fc700 1 -- 172.21.15.26:0/1626923656 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0774023e00 con 0x7f07781090b0 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.874+0000 7f077f410700 1 -- 172.21.15.26:0/1626923656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f07781a1de0 con 0x7f07781090b0 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.874+0000 7f076e7fc700 1 -- 172.21.15.26:0/1626923656 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f07781a1de0 con 0x7f07781090b0 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.878+0000 7f077f410700 1 -- 172.21.15.26:0/1626923656 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0764039510 msgr2=0x7f076403b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.878+0000 7f077f410700 1 --2- 172.21.15.26:0/1626923656 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0764039510 0x7f076403b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0768006fd0 tx=0x7f0768006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.878+0000 7f077f410700 1 -- 172.21.15.26:0/1626923656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 msgr2=0x7f077819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:26.468 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.878+0000 7f077f410700 1 --2- 172.21.15.26:0/1626923656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 0x7f077819e750 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0774009fb0 tx=0x7f0774011940 comp rx=0 tx=0).stop 2024-04-21T10:21:26.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.878+0000 7f077f410700 1 -- 172.21.15.26:0/1626923656 shutdown_connections 2024-04-21T10:21:26.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.878+0000 7f077f410700 1 --2- 172.21.15.26:0/1626923656 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0764039510 0x7f076403b9d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:26.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.878+0000 7f077f410700 1 --2- 172.21.15.26:0/1626923656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07781090b0 0x7f077819e750 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:26.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.878+0000 7f077f410700 1 -- 172.21.15.26:0/1626923656 >> 172.21.15.26:0/1626923656 conn(0x7f0778102b80 msgr2=0x7f0778103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:26.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.878+0000 7f077f410700 1 -- 172.21.15.26:0/1626923656 shutdown_connections 2024-04-21T10:21:26.469 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:25.878+0000 7f077f410700 1 -- 172.21.15.26:0/1626923656 wait complete. 2024-04-21T10:21:26.469 INFO:teuthology.orchestra.run.smithi026.stdout:mgr is available 2024-04-21T10:21:27.096 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:26 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1626923656' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T10:21:27.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:27.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [global] 2024-04-21T10:21:27.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout fsid = b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:21:27.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-21T10:21:27.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-21T10:21:27.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-21T10:21:27.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-21T10:21:27.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-21T10:21:27.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:27.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-21T10:21:27.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-21T10:21:27.644 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [osd] 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.879+0000 7f945d9dd700 1 Processor -- start 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.879+0000 7f945d9dd700 1 -- start start 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.879+0000 7f945d9dd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 0x7f945810b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.879+0000 7f945d9dd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9458063d20 con 0x7f94581090f0 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.879+0000 7f9456ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 0x7f945810b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.879+0000 7f9456ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 0x7f945810b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53022/0 (socket says 172.21.15.26:53022) 2024-04-21T10:21:27.645 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.879+0000 7f9456ffd700 1 -- 172.21.15.26:0/1196416011 learned_addr learned my addr 172.21.15.26:0/1196416011 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.880+0000 7f9456ffd700 1 -- 172.21.15.26:0/1196416011 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945810ba20 con 0x7f94581090f0 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.880+0000 7f9456ffd700 1 --2- 172.21.15.26:0/1196416011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 0x7f945810b4e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f94480098a0 tx=0x7f9448009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e4cbbde8688ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.880+0000 7f9455ffb700 1 -- 172.21.15.26:0/1196416011 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9448004160 con 0x7f94581090f0 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.880+0000 7f9455ffb700 1 -- 172.21.15.26:0/1196416011 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9448004300 con 0x7f94581090f0 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.880+0000 7f9455ffb700 1 -- 172.21.15.26:0/1196416011 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9448011af0 con 0x7f94581090f0 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.880+0000 7f945d9dd700 1 -- 172.21.15.26:0/1196416011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 msgr2=0x7f945810b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.880+0000 7f945d9dd700 1 --2- 172.21.15.26:0/1196416011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 0x7f945810b4e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f94480098a0 tx=0x7f9448009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.880+0000 7f945d9dd700 1 -- 172.21.15.26:0/1196416011 shutdown_connections 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.880+0000 7f945d9dd700 1 --2- 172.21.15.26:0/1196416011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 0x7f945810b4e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.880+0000 7f945d9dd700 1 -- 172.21.15.26:0/1196416011 >> 172.21.15.26:0/1196416011 conn(0x7f9458102b80 msgr2=0x7f9458104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.881+0000 7f945d9dd700 1 -- 172.21.15.26:0/1196416011 shutdown_connections 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.881+0000 7f945d9dd700 1 -- 172.21.15.26:0/1196416011 wait complete. 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.881+0000 7f945d9dd700 1 Processor -- start 2024-04-21T10:21:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.881+0000 7f945d9dd700 1 -- start start 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.882+0000 7f945d9dd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 0x7f9458108970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.882+0000 7f945d9dd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f945810c970 con 0x7f94581090f0 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.882+0000 7f9456ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 0x7f9458108970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.882+0000 7f9456ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 0x7f9458108970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53028/0 (socket says 172.21.15.26:53028) 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.882+0000 7f9456ffd700 1 -- 172.21.15.26:0/390228193 learned_addr learned my addr 172.21.15.26:0/390228193 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.882+0000 7f9456ffd700 1 -- 172.21.15.26:0/390228193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9458108eb0 con 0x7f94581090f0 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.882+0000 7f9456ffd700 1 --2- 172.21.15.26:0/390228193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 0x7f9458108970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9448009fb0 tx=0x7f9448011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.882+0000 7f945c9db700 1 -- 172.21.15.26:0/390228193 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94480036a0 con 0x7f94581090f0 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.882+0000 7f945c9db700 1 -- 172.21.15.26:0/390228193 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f944801a430 con 0x7f94581090f0 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.882+0000 7f945d9dd700 1 -- 172.21.15.26:0/390228193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94581070a0 con 0x7f94581090f0 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.882+0000 7f945c9db700 1 -- 172.21.15.26:0/390228193 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9448019550 con 0x7f94581090f0 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.882+0000 7f945d9dd700 1 -- 172.21.15.26:0/390228193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94581075e0 con 0x7f94581090f0 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.883+0000 7f945c9db700 1 -- 172.21.15.26:0/390228193 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f9448019800 con 0x7f94581090f0 2024-04-21T10:21:27.647 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.883+0000 7f945c9db700 1 --2- 172.21.15.26:0/390228193 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f9440039560 0x7f944003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.883+0000 7f945d9dd700 1 -- 172.21.15.26:0/390228193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94580640a0 con 0x7f94581090f0 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.883+0000 7f945c9db700 1 -- 172.21.15.26:0/390228193 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f944804d8a0 con 0x7f94581090f0 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.883+0000 7f94567fc700 1 --2- 172.21.15.26:0/390228193 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f9440039560 0x7f944003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.884+0000 7f94567fc700 1 --2- 172.21.15.26:0/390228193 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f9440039560 0x7f944003ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f944c006fd0 tx=0x7f944c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:26.889+0000 7f945c9db700 1 -- 172.21.15.26:0/390228193 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f944804d2c0 con 0x7f94581090f0 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:27.093+0000 7f945d9dd700 1 -- 172.21.15.26:0/390228193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9458107ef0 con 0x7f94581090f0 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:27.094+0000 7f945c9db700 1 -- 172.21.15.26:0/390228193 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f944800fda0 con 0x7f94581090f0 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:27.096+0000 7f945d9dd700 1 -- 172.21.15.26:0/390228193 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f9440039560 msgr2=0x7f944003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:27.096+0000 7f945d9dd700 1 --2- 172.21.15.26:0/390228193 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f9440039560 0x7f944003ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f944c006fd0 tx=0x7f944c006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:27.096+0000 7f945d9dd700 1 -- 172.21.15.26:0/390228193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 msgr2=0x7f9458108970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:27.096+0000 7f945d9dd700 1 --2- 172.21.15.26:0/390228193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 0x7f9458108970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9448009fb0 tx=0x7f9448011940 comp rx=0 tx=0).stop 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:27.096+0000 7f945d9dd700 1 -- 172.21.15.26:0/390228193 shutdown_connections 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:27.096+0000 7f945d9dd700 1 --2- 172.21.15.26:0/390228193 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f9440039560 0x7f944003ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:27.096+0000 7f945d9dd700 1 --2- 172.21.15.26:0/390228193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94581090f0 0x7f9458108970 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:27.648 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:27.096+0000 7f945d9dd700 1 -- 172.21.15.26:0/390228193 >> 172.21.15.26:0/390228193 conn(0x7f9458102b80 msgr2=0x7f9458103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:27.649 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:27.096+0000 7f945d9dd700 1 -- 172.21.15.26:0/390228193 shutdown_connections 2024-04-21T10:21:27.649 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:27.096+0000 7f945d9dd700 1 -- 172.21.15.26:0/390228193 wait complete. 2024-04-21T10:21:27.649 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling cephadm module... 2024-04-21T10:21:27.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:27 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/390228193' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-21T10:21:29.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:28 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2221901021' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-21T10:21:29.185 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:28 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: ignoring --setuser ceph since I am not root 2024-04-21T10:21:29.185 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:28 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: ignoring --setgroup ceph since I am not root 2024-04-21T10:21:29.185 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:29 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:29.057+0000 7efcddc97200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T10:21:29.665 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:29 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:29.247+0000 7efcddc97200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:21:29.665 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:29 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:29.354+0000 7efcddc97200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T10:21:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.059+0000 7fd88b583700 1 Processor -- start 2024-04-21T10:21:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.060+0000 7fd88b583700 1 -- start start 2024-04-21T10:21:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.060+0000 7fd88b583700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 0x7fd884108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.060+0000 7fd88b583700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd884109060 con 0x7fd884108670 2024-04-21T10:21:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.060+0000 7fd88931f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 0x7fd884108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.060+0000 7fd88931f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 0x7fd884108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53042/0 (socket says 172.21.15.26:53042) 2024-04-21T10:21:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.060+0000 7fd88931f700 1 -- 172.21.15.26:0/2507680576 learned_addr learned my addr 172.21.15.26:0/2507680576 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.060+0000 7fd88931f700 1 -- 172.21.15.26:0/2507680576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd884109820 con 0x7fd884108670 2024-04-21T10:21:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.061+0000 7fd88931f700 1 --2- 172.21.15.26:0/2507680576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 0x7fd884108a90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd8800098a0 tx=0x7fd880009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28cdb889112097a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.061+0000 7fd87bfff700 1 -- 172.21.15.26:0/2507680576 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd880004160 con 0x7fd884108670 2024-04-21T10:21:29.675 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.061+0000 7fd87bfff700 1 -- 172.21.15.26:0/2507680576 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd880004300 con 0x7fd884108670 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.061+0000 7fd87bfff700 1 -- 172.21.15.26:0/2507680576 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd880011af0 con 0x7fd884108670 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.061+0000 7fd88b583700 1 -- 172.21.15.26:0/2507680576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 msgr2=0x7fd884108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.061+0000 7fd88b583700 1 --2- 172.21.15.26:0/2507680576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 0x7fd884108a90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd8800098a0 tx=0x7fd880009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.061+0000 7fd88b583700 1 -- 172.21.15.26:0/2507680576 shutdown_connections 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.061+0000 7fd88b583700 1 --2- 172.21.15.26:0/2507680576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 0x7fd884108a90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.061+0000 7fd88b583700 1 -- 172.21.15.26:0/2507680576 >> 172.21.15.26:0/2507680576 conn(0x7fd884103c30 msgr2=0x7fd884106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.062+0000 7fd88b583700 1 -- 172.21.15.26:0/2507680576 shutdown_connections 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.062+0000 7fd88b583700 1 -- 172.21.15.26:0/2507680576 wait complete. 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.062+0000 7fd88b583700 1 Processor -- start 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.062+0000 7fd88b583700 1 -- start start 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.062+0000 7fd88b583700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 0x7fd88419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.062+0000 7fd88b583700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd88410ca50 con 0x7fd884108670 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.062+0000 7fd88931f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 0x7fd88419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:29.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.063+0000 7fd88931f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 0x7fd88419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53046/0 (socket says 172.21.15.26:53046) 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.063+0000 7fd88931f700 1 -- 172.21.15.26:0/2221901021 learned_addr learned my addr 172.21.15.26:0/2221901021 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.063+0000 7fd88931f700 1 -- 172.21.15.26:0/2221901021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88419ca90 con 0x7fd884108670 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.063+0000 7fd88931f700 1 --2- 172.21.15.26:0/2221901021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 0x7fd88419c550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd880009fb0 tx=0x7fd880011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.063+0000 7fd87a7fc700 1 -- 172.21.15.26:0/2221901021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8800036a0 con 0x7fd884108670 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.063+0000 7fd87a7fc700 1 -- 172.21.15.26:0/2221901021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd88001a430 con 0x7fd884108670 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.063+0000 7fd87a7fc700 1 -- 172.21.15.26:0/2221901021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd880019550 con 0x7fd884108670 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.063+0000 7fd88b583700 1 -- 172.21.15.26:0/2221901021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88419cd20 con 0x7fd884108670 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.063+0000 7fd88b583700 1 -- 172.21.15.26:0/2221901021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88419fa10 con 0x7fd884108670 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.064+0000 7fd87a7fc700 1 -- 172.21.15.26:0/2221901021 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fd880019800 con 0x7fd884108670 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.064+0000 7fd88b583700 1 -- 172.21.15.26:0/2221901021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88410cbd0 con 0x7fd884108670 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.064+0000 7fd87a7fc700 1 --2- 172.21.15.26:0/2221901021 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fd870039560 0x7fd87003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.064+0000 7fd87a7fc700 1 -- 172.21.15.26:0/2221901021 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd88004d9a0 con 0x7fd884108670 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.064+0000 7fd888b1e700 1 --2- 172.21.15.26:0/2221901021 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fd870039560 0x7fd87003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:29.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.065+0000 7fd888b1e700 1 --2- 172.21.15.26:0/2221901021 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fd870039560 0x7fd87003ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd874006fd0 tx=0x7fd874006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.070+0000 7fd87a7fc700 1 -- 172.21.15.26:0/2221901021 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd88004d3c0 con 0x7fd884108670 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.339+0000 7fd88b583700 1 -- 172.21.15.26:0/2221901021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fd8841a0060 con 0x7fd884108670 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.850+0000 7fd87a7fc700 1 -- 172.21.15.26:0/2221901021 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fd88002da20 con 0x7fd884108670 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.852+0000 7fd87a7fc700 1 -- 172.21.15.26:0/2221901021 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fd880018350 con 0x7fd884108670 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.854+0000 7fd88b583700 1 -- 172.21.15.26:0/2221901021 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fd870039560 msgr2=0x7fd87003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.854+0000 7fd88b583700 1 --2- 172.21.15.26:0/2221901021 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fd870039560 0x7fd87003ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd874006fd0 tx=0x7fd874006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.855+0000 7fd88b583700 1 -- 172.21.15.26:0/2221901021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 msgr2=0x7fd88419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.855+0000 7fd88b583700 1 --2- 172.21.15.26:0/2221901021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 0x7fd88419c550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd880009fb0 tx=0x7fd880011940 comp rx=0 tx=0).stop 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.855+0000 7fd88b583700 1 -- 172.21.15.26:0/2221901021 shutdown_connections 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.855+0000 7fd88b583700 1 --2- 172.21.15.26:0/2221901021 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fd870039560 0x7fd87003ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.855+0000 7fd88b583700 1 --2- 172.21.15.26:0/2221901021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd884108670 0x7fd88419c550 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.855+0000 7fd88b583700 1 -- 172.21.15.26:0/2221901021 >> 172.21.15.26:0/2221901021 conn(0x7fd884103c30 msgr2=0x7fd884104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.855+0000 7fd88b583700 1 -- 172.21.15.26:0/2221901021 shutdown_connections 2024-04-21T10:21:29.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:28.855+0000 7fd88b583700 1 -- 172.21.15.26:0/2221901021 wait complete. 2024-04-21T10:21:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:29 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2221901021' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-21T10:21:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:29 smithi026 ceph-mon[27965]: mgrmap e5: x(active, since 6s) 2024-04-21T10:21:29.935 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:29 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:29.673+0000 7efcddc97200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T10:21:29.936 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:29 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:29.771+0000 7efcddc97200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T10:21:30.955 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-04-21T10:21:30.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-21T10:21:30.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T10:21:30.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-21T10:21:30.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-21T10:21:30.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-04-21T10:21:30.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.085+0000 7f0d3c4ab700 1 Processor -- start 2024-04-21T10:21:30.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.085+0000 7f0d3c4ab700 1 -- start start 2024-04-21T10:21:30.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.086+0000 7f0d3c4ab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 0x7f0d34107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:30.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.086+0000 7f0d3c4ab700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0d34107f30 con 0x7f0d34107540 2024-04-21T10:21:30.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.086+0000 7f0d3a247700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 0x7f0d34107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:30.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.086+0000 7f0d3a247700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 0x7f0d34107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53076/0 (socket says 172.21.15.26:53076) 2024-04-21T10:21:30.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.086+0000 7f0d3a247700 1 -- 172.21.15.26:0/1118846738 learned_addr learned my addr 172.21.15.26:0/1118846738 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.086+0000 7f0d3a247700 1 -- 172.21.15.26:0/1118846738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d34108740 con 0x7f0d34107540 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.086+0000 7f0d3a247700 1 --2- 172.21.15.26:0/1118846738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 0x7f0d34107960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0d300098a0 tx=0x7f0d30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2de22b43ffad848 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.086+0000 7f0d39245700 1 -- 172.21.15.26:0/1118846738 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d30004160 con 0x7f0d34107540 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.086+0000 7f0d39245700 1 -- 172.21.15.26:0/1118846738 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0d30004300 con 0x7f0d34107540 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.086+0000 7f0d39245700 1 -- 172.21.15.26:0/1118846738 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d30011af0 con 0x7f0d34107540 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.087+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/1118846738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 msgr2=0x7f0d34107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.087+0000 7f0d3c4ab700 1 --2- 172.21.15.26:0/1118846738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 0x7f0d34107960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0d300098a0 tx=0x7f0d30009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.087+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/1118846738 shutdown_connections 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.087+0000 7f0d3c4ab700 1 --2- 172.21.15.26:0/1118846738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 0x7f0d34107960 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.087+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/1118846738 >> 172.21.15.26:0/1118846738 conn(0x7f0d34102b80 msgr2=0x7f0d34104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.087+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/1118846738 shutdown_connections 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.087+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/1118846738 wait complete. 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.088+0000 7f0d3c4ab700 1 Processor -- start 2024-04-21T10:21:30.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.088+0000 7f0d3c4ab700 1 -- start start 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.088+0000 7f0d3c4ab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 0x7f0d3419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.088+0000 7f0d3c4ab700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3410b970 con 0x7f0d34107540 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.088+0000 7f0d3a247700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 0x7f0d3419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.088+0000 7f0d3a247700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 0x7f0d3419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53086/0 (socket says 172.21.15.26:53086) 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.088+0000 7f0d3a247700 1 -- 172.21.15.26:0/3477360355 learned_addr learned my addr 172.21.15.26:0/3477360355 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.089+0000 7f0d3a247700 1 -- 172.21.15.26:0/3477360355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3419cb30 con 0x7f0d34107540 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.089+0000 7f0d3a247700 1 --2- 172.21.15.26:0/3477360355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 0x7f0d3419c5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009fb0 tx=0x7f0d30011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.089+0000 7f0d277fe700 1 -- 172.21.15.26:0/3477360355 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d300036a0 con 0x7f0d34107540 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.089+0000 7f0d277fe700 1 -- 172.21.15.26:0/3477360355 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0d3001a430 con 0x7f0d34107540 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.089+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/3477360355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3419cdc0 con 0x7f0d34107540 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.089+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/3477360355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3419fa60 con 0x7f0d34107540 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.089+0000 7f0d277fe700 1 -- 172.21.15.26:0/3477360355 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d30019550 con 0x7f0d34107540 2024-04-21T10:21:30.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.090+0000 7f0d277fe700 1 -- 172.21.15.26:0/3477360355 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f0d30019850 con 0x7f0d34107540 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.090+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/3477360355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3410baf0 con 0x7f0d34107540 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.090+0000 7f0d277fe700 1 --2- 172.21.15.26:0/3477360355 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0d20039560 0x7f0d2003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.090+0000 7f0d277fe700 1 -- 172.21.15.26:0/3477360355 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0d3004da50 con 0x7f0d34107540 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.090+0000 7f0d39a46700 1 -- 172.21.15.26:0/3477360355 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0d20039560 msgr2=0x7f0d2003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3874640691 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.090+0000 7f0d39a46700 1 --2- 172.21.15.26:0/3477360355 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0d20039560 0x7f0d2003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.095+0000 7f0d277fe700 1 -- 172.21.15.26:0/3477360355 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d3004d470 con 0x7f0d34107540 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.291+0000 7f0d39a46700 1 -- 172.21.15.26:0/3477360355 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0d20039560 msgr2=0x7f0d2003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3874640691 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.291+0000 7f0d39a46700 1 --2- 172.21.15.26:0/3477360355 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0d20039560 0x7f0d2003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.409+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/3477360355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f0d3402e240 con 0x7f0d34107540 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.409+0000 7f0d277fe700 1 -- 172.21.15.26:0/3477360355 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f0d3001a5e0 con 0x7f0d34107540 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.412+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/3477360355 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0d20039560 msgr2=0x7f0d2003ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.412+0000 7f0d3c4ab700 1 --2- 172.21.15.26:0/3477360355 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0d20039560 0x7f0d2003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.412+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/3477360355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 msgr2=0x7f0d3419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:30.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.412+0000 7f0d3c4ab700 1 --2- 172.21.15.26:0/3477360355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 0x7f0d3419c5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009fb0 tx=0x7f0d30011940 comp rx=0 tx=0).stop 2024-04-21T10:21:30.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.412+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/3477360355 shutdown_connections 2024-04-21T10:21:30.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.412+0000 7f0d3c4ab700 1 --2- 172.21.15.26:0/3477360355 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7f0d20039560 0x7f0d2003ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:30.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.412+0000 7f0d3c4ab700 1 --2- 172.21.15.26:0/3477360355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d34107540 0x7f0d3419c5f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:30.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.412+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/3477360355 >> 172.21.15.26:0/3477360355 conn(0x7f0d34102b80 msgr2=0x7f0d34103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:30.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.412+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/3477360355 shutdown_connections 2024-04-21T10:21:30.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:30.412+0000 7f0d3c4ab700 1 -- 172.21.15.26:0/3477360355 wait complete. 2024-04-21T10:21:30.960 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for the mgr to restart... 2024-04-21T10:21:30.960 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr epoch 5... 2024-04-21T10:21:31.046 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:30 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3477360355' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-21T10:21:31.340 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:31 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:31.046+0000 7efcddc97200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T10:21:31.340 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:31 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:31.225+0000 7efcddc97200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T10:21:31.685 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:31 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:31.406+0000 7efcddc97200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T10:21:31.978 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:31 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:31.705+0000 7efcddc97200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T10:21:31.978 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:31 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:31.800+0000 7efcddc97200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T10:21:32.435 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:32 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:32.010+0000 7efcddc97200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T10:21:32.435 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:32 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:32.209+0000 7efcddc97200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:21:33.185 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:32 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:32.832+0000 7efcddc97200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T10:21:33.577 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:33 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:33.269+0000 7efcddc97200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T10:21:33.577 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:33 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:33.368+0000 7efcddc97200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T10:21:33.577 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:33 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:33.576+0000 7efcddc97200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T10:21:34.382 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:34 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:34.116+0000 7efcddc97200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T10:21:34.382 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:34 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:34.211+0000 7efcddc97200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T10:21:34.685 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:34 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:34.462+0000 7efcddc97200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T10:21:34.685 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:34 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:34.600+0000 7efcddc97200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T10:21:35.127 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:34 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:34.697+0000 7efcddc97200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T10:21:35.127 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:34 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:34.817+0000 7efcddc97200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T10:21:35.127 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:35 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:35.127+0000 7efcddc97200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T10:21:36.050 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:35 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:35.770+0000 7efcddc97200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T10:21:36.050 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:35 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:35.997+0000 7efcddc97200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T10:21:36.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: Active manager daemon x restarted 2024-04-21T10:21:36.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: Activating manager daemon x 2024-04-21T10:21:36.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: osdmap e2: 0 total, 0 up, 0 in 2024-04-21T10:21:36.393 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: mgrmap e6: x(active, starting, since 0.00360056s) 2024-04-21T10:21:36.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:21:36.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T10:21:36.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T10:21:36.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T10:21:36.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T10:21:36.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: Manager daemon x is now available 2024-04-21T10:21:36.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' 2024-04-21T10:21:36.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' 2024-04-21T10:21:36.394 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:36 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:21:37.261 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:37 smithi026 ceph-mon[27965]: Found migration_current of "None". Setting to last migration. 2024-04-21T10:21:37.261 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:37 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:21:37.261 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:37 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T10:21:37.261 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:37 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T10:21:37.261 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:37 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' 2024-04-21T10:21:37.261 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:37 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' 2024-04-21T10:21:37.261 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:37 smithi026 ceph-mon[27965]: mgrmap e7: x(active, since 1.00547s) 2024-04-21T10:21:37.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-04-21T10:21:37.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-21T10:21:37.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-21T10:21:37.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-04-21T10:21:37.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.370+0000 7fde1b967700 1 Processor -- start 2024-04-21T10:21:37.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.370+0000 7fde1b967700 1 -- start start 2024-04-21T10:21:37.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.371+0000 7fde1b967700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 0x7fde14107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:37.479 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.371+0000 7fde1b967700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fde14107f30 con 0x7fde14107540 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.371+0000 7fde19703700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 0x7fde14107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.371+0000 7fde19703700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 0x7fde14107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53090/0 (socket says 172.21.15.26:53090) 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.371+0000 7fde19703700 1 -- 172.21.15.26:0/610556839 learned_addr learned my addr 172.21.15.26:0/610556839 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.371+0000 7fde19703700 1 -- 172.21.15.26:0/610556839 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde14108740 con 0x7fde14107540 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.371+0000 7fde19703700 1 --2- 172.21.15.26:0/610556839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 0x7fde14107960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde1000bb40 tx=0x7fde1000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9bd7644e46172b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.372+0000 7fde07fff700 1 -- 172.21.15.26:0/610556839 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde1000dbe0 con 0x7fde14107540 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.372+0000 7fde07fff700 1 -- 172.21.15.26:0/610556839 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fde100044c0 con 0x7fde14107540 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.372+0000 7fde07fff700 1 -- 172.21.15.26:0/610556839 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde10010470 con 0x7fde14107540 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.372+0000 7fde1b967700 1 -- 172.21.15.26:0/610556839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 msgr2=0x7fde14107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.372+0000 7fde1b967700 1 --2- 172.21.15.26:0/610556839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 0x7fde14107960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde1000bb40 tx=0x7fde1000be50 comp rx=0 tx=0).stop 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.372+0000 7fde1b967700 1 -- 172.21.15.26:0/610556839 shutdown_connections 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.372+0000 7fde1b967700 1 --2- 172.21.15.26:0/610556839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 0x7fde14107960 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:37.480 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.372+0000 7fde1b967700 1 -- 172.21.15.26:0/610556839 >> 172.21.15.26:0/610556839 conn(0x7fde14102b20 msgr2=0x7fde14104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:37.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.372+0000 7fde1b967700 1 -- 172.21.15.26:0/610556839 shutdown_connections 2024-04-21T10:21:37.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.372+0000 7fde1b967700 1 -- 172.21.15.26:0/610556839 wait complete. 2024-04-21T10:21:37.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.373+0000 7fde1b967700 1 Processor -- start 2024-04-21T10:21:37.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.373+0000 7fde1b967700 1 -- start start 2024-04-21T10:21:37.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.373+0000 7fde1b967700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 0x7fde1419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:37.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.373+0000 7fde1b967700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fde1407bfd0 con 0x7fde14107540 2024-04-21T10:21:37.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.374+0000 7fde19703700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 0x7fde1419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:37.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.374+0000 7fde19703700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 0x7fde1419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53096/0 (socket says 172.21.15.26:53096) 2024-04-21T10:21:37.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.374+0000 7fde19703700 1 -- 172.21.15.26:0/2733638068 learned_addr learned my addr 172.21.15.26:0/2733638068 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:37.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.374+0000 7fde19703700 1 -- 172.21.15.26:0/2733638068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1419a870 con 0x7fde14107540 2024-04-21T10:21:37.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.374+0000 7fde19703700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 0x7fde1419a330 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fde1000b340 tx=0x7fde10003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:37.481 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.374+0000 7fde067fc700 1 -- 172.21.15.26:0/2733638068 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde10027070 con 0x7fde14107540 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.374+0000 7fde067fc700 1 -- 172.21.15.26:0/2733638068 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fde10010bb0 con 0x7fde14107540 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.374+0000 7fde067fc700 1 -- 172.21.15.26:0/2733638068 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde10022ba0 con 0x7fde14107540 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.374+0000 7fde1b967700 1 -- 172.21.15.26:0/2733638068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1419ab00 con 0x7fde14107540 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.374+0000 7fde1b967700 1 -- 172.21.15.26:0/2733638068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1419d7f0 con 0x7fde14107540 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.375+0000 7fde067fc700 1 -- 172.21.15.26:0/2733638068 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fde1002a070 con 0x7fde14107540 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.376+0000 7fde067fc700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 0x7fde0003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.376+0000 7fde067fc700 1 -- 172.21.15.26:0/2733638068 --> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fde0003c1d0 con 0x7fde00039600 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.376+0000 7fde067fc700 1 -- 172.21.15.26:0/2733638068 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fde1001f070 con 0x7fde14107540 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.376+0000 7fde18f02700 1 -- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 msgr2=0x7fde0003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3874640691 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.376+0000 7fde18f02700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 0x7fde0003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.577+0000 7fde18f02700 1 -- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 msgr2=0x7fde0003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3874640691 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.577+0000 7fde18f02700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 0x7fde0003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T10:21:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.978+0000 7fde18f02700 1 -- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 msgr2=0x7fde0003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3874640691 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:31.978+0000 7fde18f02700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 0x7fde0003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:32.779+0000 7fde18f02700 1 -- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 msgr2=0x7fde0003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3874640691 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:32.779+0000 7fde18f02700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 0x7fde0003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:34.380+0000 7fde18f02700 1 -- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 msgr2=0x7fde0003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3874640691 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:34.380+0000 7fde18f02700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 0x7fde0003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:36.000+0000 7fde067fc700 1 -- 172.21.15.26:0/2733638068 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fde100183a0 con 0x7fde14107540 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:36.000+0000 7fde067fc700 1 -- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 msgr2=0x7fde0003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:36.000+0000 7fde067fc700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 0x7fde0003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.001+0000 7fde067fc700 1 -- 172.21.15.26:0/2733638068 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fde100188f0 con 0x7fde14107540 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.002+0000 7fde067fc700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fde0003cef0 0x7fde0003f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.002+0000 7fde067fc700 1 -- 172.21.15.26:0/2733638068 --> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fde10001d30 con 0x7fde0003cef0 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.002+0000 7fde18f02700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fde0003cef0 0x7fde0003f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.002+0000 7fde18f02700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fde0003cef0 0x7fde0003f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fde08003a10 tx=0x7fde080092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:37.483 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.003+0000 7fde067fc700 1 -- 172.21.15.26:0/2733638068 <== mgr.14120 v2:172.21.15.26:6800/878369551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fde10001d30 con 0x7fde0003cef0 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde1b967700 1 -- 172.21.15.26:0/2733638068 --> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fde140640a0 con 0x7fde0003cef0 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde067fc700 1 -- 172.21.15.26:0/2733638068 <== mgr.14120 v2:172.21.15.26:6800/878369551 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fde140640a0 con 0x7fde0003cef0 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde1b967700 1 -- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fde0003cef0 msgr2=0x7fde0003f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde1b967700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fde0003cef0 0x7fde0003f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fde08003a10 tx=0x7fde080092b0 comp rx=0 tx=0).stop 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde1b967700 1 -- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 msgr2=0x7fde1419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde1b967700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 0x7fde1419a330 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fde1000b340 tx=0x7fde10003dc0 comp rx=0 tx=0).stop 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde1b967700 1 -- 172.21.15.26:0/2733638068 shutdown_connections 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde1b967700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fde0003cef0 0x7fde0003f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde1b967700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde14107540 0x7fde1419a330 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde1b967700 1 --2- 172.21.15.26:0/2733638068 >> [v2:172.21.15.26:6800/3874640691,v1:172.21.15.26:6801/3874640691] conn(0x7fde00039600 0x7fde0003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde1b967700 1 -- 172.21.15.26:0/2733638068 >> 172.21.15.26:0/2733638068 conn(0x7fde14102b20 msgr2=0x7fde14190f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde1b967700 1 -- 172.21.15.26:0/2733638068 shutdown_connections 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.010+0000 7fde1b967700 1 -- 172.21.15.26:0/2733638068 wait complete. 2024-04-21T10:21:37.484 INFO:teuthology.orchestra.run.smithi026.stdout:mgr epoch 5 is available 2024-04-21T10:21:37.485 INFO:teuthology.orchestra.run.smithi026.stdout:Setting orchestrator backend to cephadm... 2024-04-21T10:21:38.121 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:38 smithi026 ceph-mon[27965]: [21/Apr/2024:10:21:36] ENGINE Bus STARTING 2024-04-21T10:21:38.122 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:38 smithi026 ceph-mon[27965]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-21T10:21:38.122 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:38 smithi026 ceph-mon[27965]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-21T10:21:38.122 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:38 smithi026 ceph-mon[27965]: [21/Apr/2024:10:21:37] ENGINE Serving on https://172.21.15.26:7150 2024-04-21T10:21:38.122 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:38 smithi026 ceph-mon[27965]: [21/Apr/2024:10:21:37] ENGINE Serving on http://172.21.15.26:8765 2024-04-21T10:21:38.122 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:38 smithi026 ceph-mon[27965]: [21/Apr/2024:10:21:37] ENGINE Bus STARTED 2024-04-21T10:21:38.123 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:38 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:21:38.715 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.885+0000 7f1663dc2700 1 Processor -- start 2024-04-21T10:21:38.715 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.886+0000 7f1663dc2700 1 -- start start 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.886+0000 7f1663dc2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 0x7f165c108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.886+0000 7f1663dc2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f165c109000 con 0x7f165c108610 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.886+0000 7f1661b5e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 0x7f165c108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.886+0000 7f1661b5e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 0x7f165c108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43456/0 (socket says 172.21.15.26:43456) 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.886+0000 7f1661b5e700 1 -- 172.21.15.26:0/484254402 learned_addr learned my addr 172.21.15.26:0/484254402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.887+0000 7f1661b5e700 1 -- 172.21.15.26:0/484254402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c109810 con 0x7f165c108610 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.887+0000 7f1661b5e700 1 --2- 172.21.15.26:0/484254402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 0x7f165c108a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f16580098a0 tx=0x7f1658009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b08c0d36b34a714 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.887+0000 7f1660b5c700 1 -- 172.21.15.26:0/484254402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1658004160 con 0x7f165c108610 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.887+0000 7f1660b5c700 1 -- 172.21.15.26:0/484254402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1658004300 con 0x7f165c108610 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.887+0000 7f1660b5c700 1 -- 172.21.15.26:0/484254402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1658011af0 con 0x7f165c108610 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.887+0000 7f1663dc2700 1 -- 172.21.15.26:0/484254402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 msgr2=0x7f165c108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.887+0000 7f1663dc2700 1 --2- 172.21.15.26:0/484254402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 0x7f165c108a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f16580098a0 tx=0x7f1658009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:38.716 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.888+0000 7f1663dc2700 1 -- 172.21.15.26:0/484254402 shutdown_connections 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.888+0000 7f1663dc2700 1 --2- 172.21.15.26:0/484254402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 0x7f165c108a30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.888+0000 7f1663dc2700 1 -- 172.21.15.26:0/484254402 >> 172.21.15.26:0/484254402 conn(0x7f165c103c10 msgr2=0x7f165c106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.888+0000 7f1663dc2700 1 -- 172.21.15.26:0/484254402 shutdown_connections 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.888+0000 7f1663dc2700 1 -- 172.21.15.26:0/484254402 wait complete. 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.888+0000 7f1663dc2700 1 Processor -- start 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.888+0000 7f1663dc2700 1 -- start start 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.888+0000 7f1663dc2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 0x7f165c07eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.889+0000 7f1663dc2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f165c10ca40 con 0x7f165c108610 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.889+0000 7f1661b5e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 0x7f165c07eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.889+0000 7f1661b5e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 0x7f165c07eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43472/0 (socket says 172.21.15.26:43472) 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.889+0000 7f1661b5e700 1 -- 172.21.15.26:0/4158306804 learned_addr learned my addr 172.21.15.26:0/4158306804 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.889+0000 7f1661b5e700 1 -- 172.21.15.26:0/4158306804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c07f0b0 con 0x7f165c108610 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.889+0000 7f1661b5e700 1 --2- 172.21.15.26:0/4158306804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 0x7f165c07eb70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16580117b0 tx=0x7f1658011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.889+0000 7f1652ffd700 1 -- 172.21.15.26:0/4158306804 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1658003af0 con 0x7f165c108610 2024-04-21T10:21:38.717 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.889+0000 7f1652ffd700 1 -- 172.21.15.26:0/4158306804 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1658020070 con 0x7f165c108610 2024-04-21T10:21:38.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.889+0000 7f1652ffd700 1 -- 172.21.15.26:0/4158306804 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1658019630 con 0x7f165c108610 2024-04-21T10:21:38.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.889+0000 7f1663dc2700 1 -- 172.21.15.26:0/4158306804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165c07d2a0 con 0x7f165c108610 2024-04-21T10:21:38.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.889+0000 7f1663dc2700 1 -- 172.21.15.26:0/4158306804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f165c07d7e0 con 0x7f165c108610 2024-04-21T10:21:38.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.890+0000 7f1652ffd700 1 -- 172.21.15.26:0/4158306804 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f16580198a0 con 0x7f165c108610 2024-04-21T10:21:38.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.890+0000 7f1663dc2700 1 -- 172.21.15.26:0/4158306804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f165c0640a0 con 0x7f165c108610 2024-04-21T10:21:38.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.890+0000 7f1652ffd700 1 --2- 172.21.15.26:0/4158306804 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f16480394e0 0x7f164803b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:38.718 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.890+0000 7f1652ffd700 1 -- 172.21.15.26:0/4158306804 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f165804dac0 con 0x7f165c108610 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.890+0000 7f166135d700 1 --2- 172.21.15.26:0/4158306804 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f16480394e0 0x7f164803b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.891+0000 7f166135d700 1 --2- 172.21.15.26:0/4158306804 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f16480394e0 0x7f164803b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f164c006fd0 tx=0x7f164c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:37.896+0000 7f1652ffd700 1 -- 172.21.15.26:0/4158306804 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16580042a0 con 0x7f165c108610 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.053+0000 7f1652ffd700 1 -- 172.21.15.26:0/4158306804 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f165800b480 con 0x7f165c108610 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.121+0000 7f1663dc2700 1 -- 172.21.15.26:0/4158306804 --> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f165c104ef0 con 0x7f16480394e0 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.126+0000 7f1652ffd700 1 -- 172.21.15.26:0/4158306804 <== mgr.14120 v2:172.21.15.26:6800/878369551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f165c104ef0 con 0x7f16480394e0 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.128+0000 7f1663dc2700 1 -- 172.21.15.26:0/4158306804 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f16480394e0 msgr2=0x7f164803b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.128+0000 7f1663dc2700 1 --2- 172.21.15.26:0/4158306804 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f16480394e0 0x7f164803b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f164c006fd0 tx=0x7f164c006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.128+0000 7f1663dc2700 1 -- 172.21.15.26:0/4158306804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 msgr2=0x7f165c07eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.128+0000 7f1663dc2700 1 --2- 172.21.15.26:0/4158306804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 0x7f165c07eb70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16580117b0 tx=0x7f1658011b70 comp rx=0 tx=0).stop 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.129+0000 7f1663dc2700 1 -- 172.21.15.26:0/4158306804 shutdown_connections 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.129+0000 7f1663dc2700 1 --2- 172.21.15.26:0/4158306804 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f16480394e0 0x7f164803b9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.129+0000 7f1663dc2700 1 --2- 172.21.15.26:0/4158306804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165c108610 0x7f165c07eb70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:38.719 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.129+0000 7f1663dc2700 1 -- 172.21.15.26:0/4158306804 >> 172.21.15.26:0/4158306804 conn(0x7f165c103c10 msgr2=0x7f165c1047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:38.720 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.129+0000 7f1663dc2700 1 -- 172.21.15.26:0/4158306804 shutdown_connections 2024-04-21T10:21:38.720 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:38.129+0000 7f1663dc2700 1 -- 172.21.15.26:0/4158306804 wait complete. 2024-04-21T10:21:39.310 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:39 smithi026 ceph-mon[27965]: mgrmap e8: x(active, since 2s) 2024-04-21T10:21:39.310 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:39 smithi026 ceph-mon[27965]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:21:39.310 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:39 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' 2024-04-21T10:21:39.310 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:39 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:21:39.897 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-21T10:21:39.897 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.088+0000 7fc32b9d5700 1 Processor -- start 2024-04-21T10:21:39.897 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.089+0000 7fc32b9d5700 1 -- start start 2024-04-21T10:21:39.897 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.089+0000 7fc32b9d5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 0x7fc32407d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:39.897 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.089+0000 7fc32b9d5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc32407d8b0 con 0x7fc32407ee80 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.089+0000 7fc329771700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 0x7fc32407d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.089+0000 7fc329771700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 0x7fc32407d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43478/0 (socket says 172.21.15.26:43478) 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.089+0000 7fc329771700 1 -- 172.21.15.26:0/2368903300 learned_addr learned my addr 172.21.15.26:0/2368903300 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.089+0000 7fc329771700 1 -- 172.21.15.26:0/2368903300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32407da30 con 0x7fc32407ee80 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.089+0000 7fc329771700 1 --2- 172.21.15.26:0/2368903300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 0x7fc32407d2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc3140098a0 tx=0x7fc314009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f50d462ea0c6f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.090+0000 7fc31bfff700 1 -- 172.21.15.26:0/2368903300 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc314004160 con 0x7fc32407ee80 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.090+0000 7fc31bfff700 1 -- 172.21.15.26:0/2368903300 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc314004300 con 0x7fc32407ee80 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.090+0000 7fc31bfff700 1 -- 172.21.15.26:0/2368903300 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc314011af0 con 0x7fc32407ee80 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.090+0000 7fc32b9d5700 1 -- 172.21.15.26:0/2368903300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 msgr2=0x7fc32407d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.090+0000 7fc32b9d5700 1 --2- 172.21.15.26:0/2368903300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 0x7fc32407d2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc3140098a0 tx=0x7fc314009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.090+0000 7fc32b9d5700 1 -- 172.21.15.26:0/2368903300 shutdown_connections 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.090+0000 7fc32b9d5700 1 --2- 172.21.15.26:0/2368903300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 0x7fc32407d2e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.090+0000 7fc32b9d5700 1 -- 172.21.15.26:0/2368903300 >> 172.21.15.26:0/2368903300 conn(0x7fc324103bf0 msgr2=0x7fc324106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.090+0000 7fc32b9d5700 1 -- 172.21.15.26:0/2368903300 shutdown_connections 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.090+0000 7fc32b9d5700 1 -- 172.21.15.26:0/2368903300 wait complete. 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.091+0000 7fc32b9d5700 1 Processor -- start 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.091+0000 7fc32b9d5700 1 -- start start 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.091+0000 7fc32b9d5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 0x7fc32419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.091+0000 7fc32b9d5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc32410eb80 con 0x7fc32407ee80 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.091+0000 7fc329771700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 0x7fc32419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.091+0000 7fc329771700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 0x7fc32419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43492/0 (socket says 172.21.15.26:43492) 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.091+0000 7fc329771700 1 -- 172.21.15.26:0/1956996748 learned_addr learned my addr 172.21.15.26:0/1956996748 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.092+0000 7fc329771700 1 -- 172.21.15.26:0/1956996748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32419ec40 con 0x7fc32407ee80 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.092+0000 7fc329771700 1 --2- 172.21.15.26:0/1956996748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 0x7fc32419e700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc314011ac0 tx=0x7fc314011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.092+0000 7fc31a7fc700 1 -- 172.21.15.26:0/1956996748 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc314003970 con 0x7fc32407ee80 2024-04-21T10:21:39.899 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.092+0000 7fc31a7fc700 1 -- 172.21.15.26:0/1956996748 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc314003b10 con 0x7fc32407ee80 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.092+0000 7fc32b9d5700 1 -- 172.21.15.26:0/1956996748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32419eed0 con 0x7fc32407ee80 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.092+0000 7fc31a7fc700 1 -- 172.21.15.26:0/1956996748 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc314024610 con 0x7fc32407ee80 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.092+0000 7fc32b9d5700 1 -- 172.21.15.26:0/1956996748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3241a1bc0 con 0x7fc32407ee80 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.093+0000 7fc31a7fc700 1 -- 172.21.15.26:0/1956996748 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc31401a430 con 0x7fc32407ee80 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.093+0000 7fc31a7fc700 1 --2- 172.21.15.26:0/1956996748 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fc310039600 0x7fc31003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.093+0000 7fc31a7fc700 1 -- 172.21.15.26:0/1956996748 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc31404d8d0 con 0x7fc32407ee80 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.093+0000 7fc32b9d5700 1 -- 172.21.15.26:0/1956996748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3240640a0 con 0x7fc32407ee80 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.093+0000 7fc328f70700 1 --2- 172.21.15.26:0/1956996748 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fc310039600 0x7fc31003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.094+0000 7fc328f70700 1 --2- 172.21.15.26:0/1956996748 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fc310039600 0x7fc31003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc320006fd0 tx=0x7fc320006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.099+0000 7fc31a7fc700 1 -- 172.21.15.26:0/1956996748 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc314016260 con 0x7fc32407ee80 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.310+0000 7fc32b9d5700 1 -- 172.21.15.26:0/1956996748 --> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fc324104e30 con 0x7fc310039600 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.311+0000 7fc31a7fc700 1 -- 172.21.15.26:0/1956996748 <== mgr.14120 v2:172.21.15.26:6800/878369551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fc324104e30 con 0x7fc310039600 2024-04-21T10:21:39.900 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.313+0000 7fc32b9d5700 1 -- 172.21.15.26:0/1956996748 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fc310039600 msgr2=0x7fc31003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:39.901 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.313+0000 7fc32b9d5700 1 --2- 172.21.15.26:0/1956996748 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fc310039600 0x7fc31003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc320006fd0 tx=0x7fc320006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:39.901 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.313+0000 7fc32b9d5700 1 -- 172.21.15.26:0/1956996748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 msgr2=0x7fc32419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:39.901 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.313+0000 7fc32b9d5700 1 --2- 172.21.15.26:0/1956996748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 0x7fc32419e700 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc314011ac0 tx=0x7fc314011ba0 comp rx=0 tx=0).stop 2024-04-21T10:21:39.901 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.313+0000 7fc32b9d5700 1 -- 172.21.15.26:0/1956996748 shutdown_connections 2024-04-21T10:21:39.901 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.313+0000 7fc32b9d5700 1 --2- 172.21.15.26:0/1956996748 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fc310039600 0x7fc31003bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:39.901 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.313+0000 7fc32b9d5700 1 --2- 172.21.15.26:0/1956996748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc32407ee80 0x7fc32419e700 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:39.901 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.313+0000 7fc32b9d5700 1 -- 172.21.15.26:0/1956996748 >> 172.21.15.26:0/1956996748 conn(0x7fc324103bf0 msgr2=0x7fc324104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:39.901 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.314+0000 7fc32b9d5700 1 -- 172.21.15.26:0/1956996748 shutdown_connections 2024-04-21T10:21:39.901 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:39.314+0000 7fc32b9d5700 1 -- 172.21.15.26:0/1956996748 wait complete. 2024-04-21T10:21:39.901 INFO:teuthology.orchestra.run.smithi026.stdout:Generating ssh key... 2024-04-21T10:21:40.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-mon[27965]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:21:40.936 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: Generating public/private rsa key pair. 2024-04-21T10:21:40.936 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: Your identification has been saved in /tmp/tmp1m6njds6/key. 2024-04-21T10:21:40.936 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: Your public key has been saved in /tmp/tmp1m6njds6/key.pub. 2024-04-21T10:21:40.936 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: The key fingerprint is: 2024-04-21T10:21:40.936 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: SHA256:JzNTqURYLZwGVm5B2yh9JLPqBcYvau7yxk9W+h4S4kA ceph-b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:21:40.936 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: The key's randomart image is: 2024-04-21T10:21:40.936 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: +---[RSA 3072]----+ 2024-04-21T10:21:40.936 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: | oBOo. | 2024-04-21T10:21:40.936 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: | o.+=X.. | 2024-04-21T10:21:40.936 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: | E =.O.= | 2024-04-21T10:21:40.936 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: | . . B + | 2024-04-21T10:21:40.937 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: | . . + S . | 2024-04-21T10:21:40.937 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: | o + * * | 2024-04-21T10:21:40.937 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: | .+ * . | 2024-04-21T10:21:40.937 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: | .ooo o . | 2024-04-21T10:21:40.937 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: | =+...o | 2024-04-21T10:21:40.937 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: +----[SHA256]-----+ 2024-04-21T10:21:41.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.309+0000 7f8d04ee8700 1 Processor -- start 2024-04-21T10:21:41.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.309+0000 7f8d04ee8700 1 -- start start 2024-04-21T10:21:41.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.310+0000 7f8d04ee8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 0x7f8d00108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:41.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.310+0000 7f8d04ee8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00109000 con 0x7f8d00108610 2024-04-21T10:21:41.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.310+0000 7f8cfe59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 0x7f8d00108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:41.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.310+0000 7f8cfe59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 0x7f8d00108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43506/0 (socket says 172.21.15.26:43506) 2024-04-21T10:21:41.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.310+0000 7f8cfe59c700 1 -- 172.21.15.26:0/623187536 learned_addr learned my addr 172.21.15.26:0/623187536 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:41.303 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.310+0000 7f8cfe59c700 1 -- 172.21.15.26:0/623187536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d00109810 con 0x7f8d00108610 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.310+0000 7f8cfe59c700 1 --2- 172.21.15.26:0/623187536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 0x7f8d00108a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40098a0 tx=0x7f8cf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d900735df74c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.311+0000 7f8cfd59a700 1 -- 172.21.15.26:0/623187536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cf4004160 con 0x7f8d00108610 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.311+0000 7f8cfd59a700 1 -- 172.21.15.26:0/623187536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8cf4004300 con 0x7f8d00108610 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.311+0000 7f8cfd59a700 1 -- 172.21.15.26:0/623187536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cf4011af0 con 0x7f8d00108610 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.311+0000 7f8d04ee8700 1 -- 172.21.15.26:0/623187536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 msgr2=0x7f8d00108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.311+0000 7f8d04ee8700 1 --2- 172.21.15.26:0/623187536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 0x7f8d00108a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40098a0 tx=0x7f8cf4009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.311+0000 7f8d04ee8700 1 -- 172.21.15.26:0/623187536 shutdown_connections 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.311+0000 7f8d04ee8700 1 --2- 172.21.15.26:0/623187536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 0x7f8d00108a30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.311+0000 7f8d04ee8700 1 -- 172.21.15.26:0/623187536 >> 172.21.15.26:0/623187536 conn(0x7f8d00103c70 msgr2=0x7f8d00106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.311+0000 7f8d04ee8700 1 -- 172.21.15.26:0/623187536 shutdown_connections 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.311+0000 7f8d04ee8700 1 -- 172.21.15.26:0/623187536 wait complete. 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.312+0000 7f8d04ee8700 1 Processor -- start 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.312+0000 7f8d04ee8700 1 -- start start 2024-04-21T10:21:41.304 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.312+0000 7f8d04ee8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 0x7f8d0019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.312+0000 7f8d04ee8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0010ca40 con 0x7f8d00108610 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.312+0000 7f8cfe59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 0x7f8d0019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.313+0000 7f8cfe59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 0x7f8d0019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43514/0 (socket says 172.21.15.26:43514) 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.313+0000 7f8cfe59c700 1 -- 172.21.15.26:0/797050686 learned_addr learned my addr 172.21.15.26:0/797050686 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.313+0000 7f8cfe59c700 1 -- 172.21.15.26:0/797050686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0019ca80 con 0x7f8d00108610 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.313+0000 7f8cfe59c700 1 --2- 172.21.15.26:0/797050686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 0x7f8d0019c540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009fb0 tx=0x7f8cf4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.313+0000 7f8cef7fe700 1 -- 172.21.15.26:0/797050686 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cf40036a0 con 0x7f8d00108610 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.313+0000 7f8cef7fe700 1 -- 172.21.15.26:0/797050686 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8cf401a430 con 0x7f8d00108610 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.313+0000 7f8d04ee8700 1 -- 172.21.15.26:0/797050686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0019cd10 con 0x7f8d00108610 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.313+0000 7f8cef7fe700 1 -- 172.21.15.26:0/797050686 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cf4019550 con 0x7f8d00108610 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.313+0000 7f8d04ee8700 1 -- 172.21.15.26:0/797050686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0019fa00 con 0x7f8d00108610 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.314+0000 7f8cef7fe700 1 -- 172.21.15.26:0/797050686 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8cf4019800 con 0x7f8d00108610 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.314+0000 7f8d04ee8700 1 -- 172.21.15.26:0/797050686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0010cbc0 con 0x7f8d00108610 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.314+0000 7f8cef7fe700 1 --2- 172.21.15.26:0/797050686 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f8ce80395b0 0x7f8ce803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:41.305 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.314+0000 7f8cef7fe700 1 -- 172.21.15.26:0/797050686 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8cf404d940 con 0x7f8d00108610 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.314+0000 7f8cfdd9b700 1 --2- 172.21.15.26:0/797050686 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f8ce80395b0 0x7f8ce803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.315+0000 7f8cfdd9b700 1 --2- 172.21.15.26:0/797050686 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f8ce80395b0 0x7f8ce803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0006fd0 tx=0x7f8cf0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.319+0000 7f8cef7fe700 1 -- 172.21.15.26:0/797050686 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8cf401a690 con 0x7f8d00108610 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.529+0000 7f8d04ee8700 1 -- 172.21.15.26:0/797050686 --> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8d00104ec0 con 0x7f8ce80395b0 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.744+0000 7f8cef7fe700 1 -- 172.21.15.26:0/797050686 <== mgr.14120 v2:172.21.15.26:6800/878369551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8d00104ec0 con 0x7f8ce80395b0 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.746+0000 7f8d04ee8700 1 -- 172.21.15.26:0/797050686 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f8ce80395b0 msgr2=0x7f8ce803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.746+0000 7f8d04ee8700 1 --2- 172.21.15.26:0/797050686 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f8ce80395b0 0x7f8ce803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0006fd0 tx=0x7f8cf0006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.746+0000 7f8d04ee8700 1 -- 172.21.15.26:0/797050686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 msgr2=0x7f8d0019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.746+0000 7f8d04ee8700 1 --2- 172.21.15.26:0/797050686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 0x7f8d0019c540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009fb0 tx=0x7f8cf4011940 comp rx=0 tx=0).stop 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.747+0000 7f8d04ee8700 1 -- 172.21.15.26:0/797050686 shutdown_connections 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.747+0000 7f8d04ee8700 1 --2- 172.21.15.26:0/797050686 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f8ce80395b0 0x7f8ce803ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.747+0000 7f8d04ee8700 1 --2- 172.21.15.26:0/797050686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d00108610 0x7f8d0019c540 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:41.306 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.747+0000 7f8d04ee8700 1 -- 172.21.15.26:0/797050686 >> 172.21.15.26:0/797050686 conn(0x7f8d00103c70 msgr2=0x7f8d001047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:41.307 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.747+0000 7f8d04ee8700 1 -- 172.21.15.26:0/797050686 shutdown_connections 2024-04-21T10:21:41.307 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:40.747+0000 7f8d04ee8700 1 -- 172.21.15.26:0/797050686 wait complete. 2024-04-21T10:21:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:41 smithi026 ceph-mon[27965]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:21:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:41 smithi026 ceph-mon[27965]: Generating ssh key... 2024-04-21T10:21:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:41 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' 2024-04-21T10:21:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:41 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' 2024-04-21T10:21:42.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPRhQDwVyQytoBWYQAnXL/X2ytGQgq/Fv6haEcFIxCmRLyCk2kw4tqgM4YQlToXJmnEytBSCLIKhqk63AsN2VqlrZIo5U3UXy+Hj75dJVgf8NfRu+q/y4bpbmBIjbcsdCaBI6/4R9doeodvs2ZiVddqpNGsFLixG8bywiadrgAS8/J+Qke++wOqTPAPNkj5kIHg0XXk5SsC1SotBC2VG5hkp268+CVKFH4Rrci89D6rf4LGrJMHbUUe2h3tvbmry5n25eQXRBZTiOG0Gw4txefr5TWpgixKfBWahEwQFIE1swpSdKjpn01szpSUnYl/6/4e5pmxUmD5D+Tm3tfqiFM3F1kWtR9oXwD0bjvBgvgbuDXEDAhBVZ9xaJyGQT2C+F8g6oYrLUN0+PnL3LvF5m5qs88Wvbnzf2Nvrj75WT1vSeXi6iLQ4b9LGNWQLcto34V7tukuMAqxp9Dgyp+a1ZlwRD14iPAi6tVqls4L0hS0LkZcaWzGsgXfHF3CpBU9KM= ceph-b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:21:42.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.730+0000 7f2affe52700 1 Processor -- start 2024-04-21T10:21:42.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.730+0000 7f2affe52700 1 -- start start 2024-04-21T10:21:42.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.730+0000 7f2affe52700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 0x7f2af8108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:42.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.730+0000 7f2affe52700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81095a0 con 0x7f2af8108bb0 2024-04-21T10:21:42.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.731+0000 7f2afdbee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 0x7f2af8108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:42.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.731+0000 7f2afdbee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 0x7f2af8108fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43516/0 (socket says 172.21.15.26:43516) 2024-04-21T10:21:42.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.731+0000 7f2afdbee700 1 -- 172.21.15.26:0/2991816510 learned_addr learned my addr 172.21.15.26:0/2991816510 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:42.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.731+0000 7f2afdbee700 1 -- 172.21.15.26:0/2991816510 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af81066b0 con 0x7f2af8108bb0 2024-04-21T10:21:42.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.731+0000 7f2afdbee700 1 --2- 172.21.15.26:0/2991816510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 0x7f2af8108fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2af40098a0 tx=0x7f2af4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd46e425fd59d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:42.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.731+0000 7f2afcbec700 1 -- 172.21.15.26:0/2991816510 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2af4004160 con 0x7f2af8108bb0 2024-04-21T10:21:42.509 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.731+0000 7f2afcbec700 1 -- 172.21.15.26:0/2991816510 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2af4004300 con 0x7f2af8108bb0 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.731+0000 7f2afcbec700 1 -- 172.21.15.26:0/2991816510 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2af4011af0 con 0x7f2af8108bb0 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.732+0000 7f2affe52700 1 -- 172.21.15.26:0/2991816510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 msgr2=0x7f2af8108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.732+0000 7f2affe52700 1 --2- 172.21.15.26:0/2991816510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 0x7f2af8108fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2af40098a0 tx=0x7f2af4009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.732+0000 7f2affe52700 1 -- 172.21.15.26:0/2991816510 shutdown_connections 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.732+0000 7f2affe52700 1 --2- 172.21.15.26:0/2991816510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 0x7f2af8108fd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.732+0000 7f2affe52700 1 -- 172.21.15.26:0/2991816510 >> 172.21.15.26:0/2991816510 conn(0x7f2af8102270 msgr2=0x7f2af8104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.732+0000 7f2affe52700 1 -- 172.21.15.26:0/2991816510 shutdown_connections 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.732+0000 7f2affe52700 1 -- 172.21.15.26:0/2991816510 wait complete. 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.733+0000 7f2affe52700 1 Processor -- start 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.733+0000 7f2affe52700 1 -- start start 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.733+0000 7f2affe52700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 0x7f2af819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.733+0000 7f2affe52700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2af8107660 con 0x7f2af8108bb0 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.733+0000 7f2afdbee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 0x7f2af819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:42.510 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.733+0000 7f2afdbee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 0x7f2af819e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43520/0 (socket says 172.21.15.26:43520) 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.733+0000 7f2afdbee700 1 -- 172.21.15.26:0/1546518700 learned_addr learned my addr 172.21.15.26:0/1546518700 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.733+0000 7f2afdbee700 1 -- 172.21.15.26:0/1546518700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af819eca0 con 0x7f2af8108bb0 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.733+0000 7f2afdbee700 1 --2- 172.21.15.26:0/1546518700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 0x7f2af819e760 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009fb0 tx=0x7f2af4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.734+0000 7f2aeeffd700 1 -- 172.21.15.26:0/1546518700 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2af40036a0 con 0x7f2af8108bb0 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.734+0000 7f2aeeffd700 1 -- 172.21.15.26:0/1546518700 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2af401a430 con 0x7f2af8108bb0 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.734+0000 7f2affe52700 1 -- 172.21.15.26:0/1546518700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af819ef30 con 0x7f2af8108bb0 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.734+0000 7f2aeeffd700 1 -- 172.21.15.26:0/1546518700 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2af4019550 con 0x7f2af8108bb0 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.734+0000 7f2affe52700 1 -- 172.21.15.26:0/1546518700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af81a1c20 con 0x7f2af8108bb0 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.735+0000 7f2aeeffd700 1 -- 172.21.15.26:0/1546518700 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2af4019850 con 0x7f2af8108bb0 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.735+0000 7f2affe52700 1 -- 172.21.15.26:0/1546518700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af80640a0 con 0x7f2af8108bb0 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.735+0000 7f2aeeffd700 1 --2- 172.21.15.26:0/1546518700 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f2ae4039560 0x7f2ae403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.735+0000 7f2aeeffd700 1 -- 172.21.15.26:0/1546518700 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2af404dab0 con 0x7f2af8108bb0 2024-04-21T10:21:42.511 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.735+0000 7f2afd3ed700 1 --2- 172.21.15.26:0/1546518700 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f2ae4039560 0x7f2ae403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.735+0000 7f2afd3ed700 1 --2- 172.21.15.26:0/1546518700 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f2ae4039560 0x7f2ae403ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8006fd0 tx=0x7f2ae8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.740+0000 7f2aeeffd700 1 -- 172.21.15.26:0/1546518700 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2af402de00 con 0x7f2af8108bb0 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.948+0000 7f2affe52700 1 -- 172.21.15.26:0/1546518700 --> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f2af80754c0 con 0x7f2ae4039560 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.948+0000 7f2aeeffd700 1 -- 172.21.15.26:0/1546518700 <== mgr.14120 v2:172.21.15.26:6800/878369551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f2af80754c0 con 0x7f2ae4039560 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.951+0000 7f2affe52700 1 -- 172.21.15.26:0/1546518700 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f2ae4039560 msgr2=0x7f2ae403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.951+0000 7f2affe52700 1 --2- 172.21.15.26:0/1546518700 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f2ae4039560 0x7f2ae403ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8006fd0 tx=0x7f2ae8006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.951+0000 7f2affe52700 1 -- 172.21.15.26:0/1546518700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 msgr2=0x7f2af819e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.951+0000 7f2affe52700 1 --2- 172.21.15.26:0/1546518700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 0x7f2af819e760 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009fb0 tx=0x7f2af4011940 comp rx=0 tx=0).stop 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.951+0000 7f2affe52700 1 -- 172.21.15.26:0/1546518700 shutdown_connections 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.951+0000 7f2affe52700 1 --2- 172.21.15.26:0/1546518700 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f2ae4039560 0x7f2ae403ba20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.951+0000 7f2affe52700 1 --2- 172.21.15.26:0/1546518700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2af8108bb0 0x7f2af819e760 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.951+0000 7f2affe52700 1 -- 172.21.15.26:0/1546518700 >> 172.21.15.26:0/1546518700 conn(0x7f2af8102270 msgr2=0x7f2af8194e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.951+0000 7f2affe52700 1 -- 172.21.15.26:0/1546518700 shutdown_connections 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:41.951+0000 7f2affe52700 1 -- 172.21.15.26:0/1546518700 wait complete. 2024-04-21T10:21:42.512 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-21T10:21:42.513 INFO:teuthology.orchestra.run.smithi026.stdout:Adding key to root@localhost authorized_keys... 2024-04-21T10:21:42.513 INFO:teuthology.orchestra.run.smithi026.stdout:Adding host smithi026... 2024-04-21T10:21:43.223 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:42 smithi026 ceph-mon[27965]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:21:44.044 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:43 smithi026 ceph-mon[27965]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi026", "addr": "172.21.15.26", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:21:44.044 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:43 smithi026 ceph-mon[27965]: Deploying cephadm binary to smithi026 2024-04-21T10:21:48.016 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Added host 'smithi026' with addr '172.21.15.26' 2024-04-21T10:21:48.016 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.923+0000 7f3db3f62700 1 Processor -- start 2024-04-21T10:21:48.016 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.924+0000 7f3db3f62700 1 -- start start 2024-04-21T10:21:48.016 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.924+0000 7f3db3f62700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 0x7f3dac10b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:48.016 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.924+0000 7f3db3f62700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac07ce50 con 0x7f3dac1090c0 2024-04-21T10:21:48.016 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.924+0000 7f3db1cfe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 0x7f3dac10b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:48.016 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.924+0000 7f3db1cfe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 0x7f3dac10b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43526/0 (socket says 172.21.15.26:43526) 2024-04-21T10:21:48.016 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.924+0000 7f3db1cfe700 1 -- 172.21.15.26:0/4216017033 learned_addr learned my addr 172.21.15.26:0/4216017033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:48.016 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.924+0000 7f3db1cfe700 1 -- 172.21.15.26:0/4216017033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac10b9f0 con 0x7f3dac1090c0 2024-04-21T10:21:48.016 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.924+0000 7f3db1cfe700 1 --2- 172.21.15.26:0/4216017033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 0x7f3dac10b4b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3da80098a0 tx=0x7f3da8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591dd62187346c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.924+0000 7f3db0cfc700 1 -- 172.21.15.26:0/4216017033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3da8004160 con 0x7f3dac1090c0 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.925+0000 7f3db0cfc700 1 -- 172.21.15.26:0/4216017033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3da8004300 con 0x7f3dac1090c0 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.925+0000 7f3db0cfc700 1 -- 172.21.15.26:0/4216017033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3da8011af0 con 0x7f3dac1090c0 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.925+0000 7f3db3f62700 1 -- 172.21.15.26:0/4216017033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 msgr2=0x7f3dac10b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.925+0000 7f3db3f62700 1 --2- 172.21.15.26:0/4216017033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 0x7f3dac10b4b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3da80098a0 tx=0x7f3da8009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.925+0000 7f3db3f62700 1 -- 172.21.15.26:0/4216017033 shutdown_connections 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.925+0000 7f3db3f62700 1 --2- 172.21.15.26:0/4216017033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 0x7f3dac10b4b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.925+0000 7f3db3f62700 1 -- 172.21.15.26:0/4216017033 >> 172.21.15.26:0/4216017033 conn(0x7f3dac102af0 msgr2=0x7f3dac104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.925+0000 7f3db3f62700 1 -- 172.21.15.26:0/4216017033 shutdown_connections 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.925+0000 7f3db3f62700 1 -- 172.21.15.26:0/4216017033 wait complete. 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.926+0000 7f3db3f62700 1 Processor -- start 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.926+0000 7f3db3f62700 1 -- start start 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.926+0000 7f3db3f62700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 0x7f3dac19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:48.017 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.926+0000 7f3db3f62700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac10c8f0 con 0x7f3dac1090c0 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.926+0000 7f3db1cfe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 0x7f3dac19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.926+0000 7f3db1cfe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 0x7f3dac19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43528/0 (socket says 172.21.15.26:43528) 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.926+0000 7f3db1cfe700 1 -- 172.21.15.26:0/1208070617 learned_addr learned my addr 172.21.15.26:0/1208070617 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.926+0000 7f3db1cfe700 1 -- 172.21.15.26:0/1208070617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac19ca60 con 0x7f3dac1090c0 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.927+0000 7f3db1cfe700 1 --2- 172.21.15.26:0/1208070617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 0x7f3dac19c520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3da8011ac0 tx=0x7f3da8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.927+0000 7f3da2ffd700 1 -- 172.21.15.26:0/1208070617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3da8003970 con 0x7f3dac1090c0 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.927+0000 7f3db3f62700 1 -- 172.21.15.26:0/1208070617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dac19ccf0 con 0x7f3dac1090c0 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.927+0000 7f3da2ffd700 1 -- 172.21.15.26:0/1208070617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3da8003b10 con 0x7f3dac1090c0 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.927+0000 7f3da2ffd700 1 -- 172.21.15.26:0/1208070617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3da8024610 con 0x7f3dac1090c0 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.927+0000 7f3db3f62700 1 -- 172.21.15.26:0/1208070617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dac19f9e0 con 0x7f3dac1090c0 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.928+0000 7f3da2ffd700 1 -- 172.21.15.26:0/1208070617 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3da801a430 con 0x7f3dac1090c0 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.928+0000 7f3da2ffd700 1 --2- 172.21.15.26:0/1208070617 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f3d980395b0 0x7f3d9803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.928+0000 7f3db3f62700 1 -- 172.21.15.26:0/1208070617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dac0640a0 con 0x7f3dac1090c0 2024-04-21T10:21:48.018 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.928+0000 7f3da2ffd700 1 -- 172.21.15.26:0/1208070617 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3da804d910 con 0x7f3dac1090c0 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.928+0000 7f3db14fd700 1 --2- 172.21.15.26:0/1208070617 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f3d980395b0 0x7f3d9803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.928+0000 7f3db14fd700 1 --2- 172.21.15.26:0/1208070617 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f3d980395b0 0x7f3d9803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c006fd0 tx=0x7f3d9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:42.933+0000 7f3da2ffd700 1 -- 172.21.15.26:0/1208070617 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3da8016260 con 0x7f3dac1090c0 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:43.147+0000 7f3db3f62700 1 -- 172.21.15.26:0/1208070617 --> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi026", "addr": "172.21.15.26", "target": ["mon-mgr", ""]}) -- 0x7f3dac103de0 con 0x7f3d980395b0 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:47.416+0000 7f3da2ffd700 1 -- 172.21.15.26:0/1208070617 <== mgr.14120 v2:172.21.15.26:6800/878369551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f3dac103de0 con 0x7f3d980395b0 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:47.419+0000 7f3db3f62700 1 -- 172.21.15.26:0/1208070617 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f3d980395b0 msgr2=0x7f3d9803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:47.419+0000 7f3db3f62700 1 --2- 172.21.15.26:0/1208070617 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f3d980395b0 0x7f3d9803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c006fd0 tx=0x7f3d9c006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:47.419+0000 7f3db3f62700 1 -- 172.21.15.26:0/1208070617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 msgr2=0x7f3dac19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:47.419+0000 7f3db3f62700 1 --2- 172.21.15.26:0/1208070617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 0x7f3dac19c520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3da8011ac0 tx=0x7f3da8011ba0 comp rx=0 tx=0).stop 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:47.419+0000 7f3db3f62700 1 -- 172.21.15.26:0/1208070617 shutdown_connections 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:47.419+0000 7f3db3f62700 1 --2- 172.21.15.26:0/1208070617 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f3d980395b0 0x7f3d9803ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:47.419+0000 7f3db3f62700 1 --2- 172.21.15.26:0/1208070617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3dac1090c0 0x7f3dac19c520 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:48.019 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:47.419+0000 7f3db3f62700 1 -- 172.21.15.26:0/1208070617 >> 172.21.15.26:0/1208070617 conn(0x7f3dac102af0 msgr2=0x7f3dac1036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:48.020 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:47.420+0000 7f3db3f62700 1 -- 172.21.15.26:0/1208070617 shutdown_connections 2024-04-21T10:21:48.020 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:47.420+0000 7f3db3f62700 1 -- 172.21.15.26:0/1208070617 wait complete. 2024-04-21T10:21:48.020 INFO:teuthology.orchestra.run.smithi026.stdout:Deploying unmanaged mon service... 2024-04-21T10:21:48.586 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:48 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' 2024-04-21T10:21:48.586 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:48 smithi026 ceph-mon[27965]: Added host smithi026 2024-04-21T10:21:48.586 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:48 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:21:49.358 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-21T10:21:49.358 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.600+0000 7f4a4aff7700 1 Processor -- start 2024-04-21T10:21:49.358 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.600+0000 7f4a4aff7700 1 -- start start 2024-04-21T10:21:49.358 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.600+0000 7f4a4aff7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 0x7f4a44108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:49.358 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.600+0000 7f4a4aff7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44109060 con 0x7f4a44108670 2024-04-21T10:21:49.358 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.600+0000 7f4a48d93700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 0x7f4a44108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:49.358 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.600+0000 7f4a48d93700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 0x7f4a44108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42986/0 (socket says 172.21.15.26:42986) 2024-04-21T10:21:49.358 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.600+0000 7f4a48d93700 1 -- 172.21.15.26:0/1768992970 learned_addr learned my addr 172.21.15.26:0/1768992970 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:49.358 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.600+0000 7f4a48d93700 1 -- 172.21.15.26:0/1768992970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a44109870 con 0x7f4a44108670 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.601+0000 7f4a48d93700 1 --2- 172.21.15.26:0/1768992970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 0x7f4a44108a90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4a380098a0 tx=0x7f4a38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca671bef451cc030 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.601+0000 7f4a437fe700 1 -- 172.21.15.26:0/1768992970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a38004160 con 0x7f4a44108670 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.601+0000 7f4a437fe700 1 -- 172.21.15.26:0/1768992970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4a38004300 con 0x7f4a44108670 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.601+0000 7f4a437fe700 1 -- 172.21.15.26:0/1768992970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a38011af0 con 0x7f4a44108670 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.601+0000 7f4a4aff7700 1 -- 172.21.15.26:0/1768992970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 msgr2=0x7f4a44108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.601+0000 7f4a4aff7700 1 --2- 172.21.15.26:0/1768992970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 0x7f4a44108a90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4a380098a0 tx=0x7f4a38009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.601+0000 7f4a4aff7700 1 -- 172.21.15.26:0/1768992970 shutdown_connections 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.601+0000 7f4a4aff7700 1 --2- 172.21.15.26:0/1768992970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 0x7f4a44108a90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.601+0000 7f4a4aff7700 1 -- 172.21.15.26:0/1768992970 >> 172.21.15.26:0/1768992970 conn(0x7f4a44103c30 msgr2=0x7f4a44106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.601+0000 7f4a4aff7700 1 -- 172.21.15.26:0/1768992970 shutdown_connections 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.602+0000 7f4a4aff7700 1 -- 172.21.15.26:0/1768992970 wait complete. 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.602+0000 7f4a4aff7700 1 Processor -- start 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.602+0000 7f4a4aff7700 1 -- start start 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.602+0000 7f4a4aff7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 0x7f4a4419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.602+0000 7f4a4aff7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4410caa0 con 0x7f4a44108670 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.602+0000 7f4a48d93700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 0x7f4a4419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.602+0000 7f4a48d93700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 0x7f4a4419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42998/0 (socket says 172.21.15.26:42998) 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.602+0000 7f4a48d93700 1 -- 172.21.15.26:0/3176395118 learned_addr learned my addr 172.21.15.26:0/3176395118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.603+0000 7f4a48d93700 1 -- 172.21.15.26:0/3176395118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4419caa0 con 0x7f4a44108670 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.603+0000 7f4a48d93700 1 --2- 172.21.15.26:0/3176395118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 0x7f4a4419c560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009fb0 tx=0x7f4a38011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.603+0000 7f4a41ffb700 1 -- 172.21.15.26:0/3176395118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a380036a0 con 0x7f4a44108670 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.603+0000 7f4a41ffb700 1 -- 172.21.15.26:0/3176395118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4a3801a430 con 0x7f4a44108670 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.603+0000 7f4a4aff7700 1 -- 172.21.15.26:0/3176395118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4419cd30 con 0x7f4a44108670 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.603+0000 7f4a41ffb700 1 -- 172.21.15.26:0/3176395118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a38019550 con 0x7f4a44108670 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.603+0000 7f4a4aff7700 1 -- 172.21.15.26:0/3176395118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4419fa20 con 0x7f4a44108670 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.604+0000 7f4a41ffb700 1 -- 172.21.15.26:0/3176395118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4a38019850 con 0x7f4a44108670 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.604+0000 7f4a4aff7700 1 -- 172.21.15.26:0/3176395118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a440640a0 con 0x7f4a44108670 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.604+0000 7f4a41ffb700 1 --2- 172.21.15.26:0/3176395118 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f4a34039560 0x7f4a3403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:49.360 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.604+0000 7f4a41ffb700 1 -- 172.21.15.26:0/3176395118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4a3804d910 con 0x7f4a44108670 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.604+0000 7f4a43fff700 1 --2- 172.21.15.26:0/3176395118 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f4a34039560 0x7f4a3403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.605+0000 7f4a43fff700 1 --2- 172.21.15.26:0/3176395118 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f4a34039560 0x7f4a3403ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4a30006fd0 tx=0x7f4a30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.609+0000 7f4a41ffb700 1 -- 172.21.15.26:0/3176395118 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a3800fda0 con 0x7f4a44108670 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.833+0000 7f4a4aff7700 1 -- 172.21.15.26:0/3176395118 --> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4a44104eb0 con 0x7f4a34039560 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.837+0000 7f4a41ffb700 1 -- 172.21.15.26:0/3176395118 <== mgr.14120 v2:172.21.15.26:6800/878369551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4a44104eb0 con 0x7f4a34039560 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.839+0000 7f4a4aff7700 1 -- 172.21.15.26:0/3176395118 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f4a34039560 msgr2=0x7f4a3403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.839+0000 7f4a4aff7700 1 --2- 172.21.15.26:0/3176395118 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f4a34039560 0x7f4a3403ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4a30006fd0 tx=0x7f4a30006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.839+0000 7f4a4aff7700 1 -- 172.21.15.26:0/3176395118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 msgr2=0x7f4a4419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.839+0000 7f4a4aff7700 1 --2- 172.21.15.26:0/3176395118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 0x7f4a4419c560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009fb0 tx=0x7f4a38011940 comp rx=0 tx=0).stop 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.840+0000 7f4a4aff7700 1 -- 172.21.15.26:0/3176395118 shutdown_connections 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.840+0000 7f4a4aff7700 1 --2- 172.21.15.26:0/3176395118 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f4a34039560 0x7f4a3403ba20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.840+0000 7f4a4aff7700 1 --2- 172.21.15.26:0/3176395118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a44108670 0x7f4a4419c560 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.840+0000 7f4a4aff7700 1 -- 172.21.15.26:0/3176395118 >> 172.21.15.26:0/3176395118 conn(0x7f4a44103c30 msgr2=0x7f4a441047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.840+0000 7f4a4aff7700 1 -- 172.21.15.26:0/3176395118 shutdown_connections 2024-04-21T10:21:49.361 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:48.840+0000 7f4a4aff7700 1 -- 172.21.15.26:0/3176395118 wait complete. 2024-04-21T10:21:49.362 INFO:teuthology.orchestra.run.smithi026.stdout:Deploying unmanaged mgr service... 2024-04-21T10:21:49.997 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:49 smithi026 ceph-mon[27965]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:21:49.998 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:49 smithi026 ceph-mon[27965]: Saving service mon spec with placement count:5 2024-04-21T10:21:49.998 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:49 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' 2024-04-21T10:21:50.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-21T10:21:50.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.767+0000 7f76eb48b700 1 Processor -- start 2024-04-21T10:21:50.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.767+0000 7f76eb48b700 1 -- start start 2024-04-21T10:21:50.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.767+0000 7f76eb48b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 0x7f76e407d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:50.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.767+0000 7f76eb48b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76e407d910 con 0x7f76e407eee0 2024-04-21T10:21:50.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.767+0000 7f76e9227700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 0x7f76e407d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:50.797 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.767+0000 7f76e9227700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 0x7f76e407d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43010/0 (socket says 172.21.15.26:43010) 2024-04-21T10:21:50.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.768+0000 7f76e9227700 1 -- 172.21.15.26:0/2786496658 learned_addr learned my addr 172.21.15.26:0/2786496658 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:50.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.768+0000 7f76e9227700 1 -- 172.21.15.26:0/2786496658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e407da90 con 0x7f76e407eee0 2024-04-21T10:21:50.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.768+0000 7f76e9227700 1 --2- 172.21.15.26:0/2786496658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 0x7f76e407d340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76e00098a0 tx=0x7f76e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa88f2727fc631c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:50.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.768+0000 7f76dbfff700 1 -- 172.21.15.26:0/2786496658 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76e0004160 con 0x7f76e407eee0 2024-04-21T10:21:50.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.768+0000 7f76dbfff700 1 -- 172.21.15.26:0/2786496658 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f76e0004300 con 0x7f76e407eee0 2024-04-21T10:21:50.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.768+0000 7f76dbfff700 1 -- 172.21.15.26:0/2786496658 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76e0011af0 con 0x7f76e407eee0 2024-04-21T10:21:50.798 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.769+0000 7f76eb48b700 1 -- 172.21.15.26:0/2786496658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 msgr2=0x7f76e407d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.769+0000 7f76eb48b700 1 --2- 172.21.15.26:0/2786496658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 0x7f76e407d340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76e00098a0 tx=0x7f76e0009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.769+0000 7f76eb48b700 1 -- 172.21.15.26:0/2786496658 shutdown_connections 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.769+0000 7f76eb48b700 1 --2- 172.21.15.26:0/2786496658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 0x7f76e407d340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.769+0000 7f76eb48b700 1 -- 172.21.15.26:0/2786496658 >> 172.21.15.26:0/2786496658 conn(0x7f76e4103bc0 msgr2=0x7f76e4105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.769+0000 7f76eb48b700 1 -- 172.21.15.26:0/2786496658 shutdown_connections 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.769+0000 7f76eb48b700 1 -- 172.21.15.26:0/2786496658 wait complete. 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.769+0000 7f76eb48b700 1 Processor -- start 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.770+0000 7f76eb48b700 1 -- start start 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.770+0000 7f76eb48b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 0x7f76e419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.770+0000 7f76eb48b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76e410eb60 con 0x7f76e407eee0 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.770+0000 7f76e9227700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 0x7f76e419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.770+0000 7f76e9227700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 0x7f76e419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43016/0 (socket says 172.21.15.26:43016) 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.770+0000 7f76e9227700 1 -- 172.21.15.26:0/3608765724 learned_addr learned my addr 172.21.15.26:0/3608765724 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:50.799 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.770+0000 7f76e9227700 1 -- 172.21.15.26:0/3608765724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e419ed00 con 0x7f76e407eee0 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.770+0000 7f76e9227700 1 --2- 172.21.15.26:0/3608765724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 0x7f76e419e7c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76e00117b0 tx=0x7f76e0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.770+0000 7f76da7fc700 1 -- 172.21.15.26:0/3608765724 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76e0003af0 con 0x7f76e407eee0 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.770+0000 7f76da7fc700 1 -- 172.21.15.26:0/3608765724 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f76e0020070 con 0x7f76e407eee0 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.770+0000 7f76eb48b700 1 -- 172.21.15.26:0/3608765724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e419ef90 con 0x7f76e407eee0 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.770+0000 7f76da7fc700 1 -- 172.21.15.26:0/3608765724 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76e0019550 con 0x7f76e407eee0 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.771+0000 7f76eb48b700 1 -- 172.21.15.26:0/3608765724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e41a1c80 con 0x7f76e407eee0 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.771+0000 7f76da7fc700 1 -- 172.21.15.26:0/3608765724 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f76e001a430 con 0x7f76e407eee0 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.772+0000 7f76eb48b700 1 -- 172.21.15.26:0/3608765724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e40640a0 con 0x7f76e407eee0 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.772+0000 7f76da7fc700 1 --2- 172.21.15.26:0/3608765724 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f76d0039660 0x7f76d003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.772+0000 7f76da7fc700 1 -- 172.21.15.26:0/3608765724 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f76e004dab0 con 0x7f76e407eee0 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.772+0000 7f76e8a26700 1 --2- 172.21.15.26:0/3608765724 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f76d0039660 0x7f76d003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.772+0000 7f76e8a26700 1 --2- 172.21.15.26:0/3608765724 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f76d0039660 0x7f76d003bb20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f76d4006fd0 tx=0x7f76d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:50.800 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.777+0000 7f76da7fc700 1 -- 172.21.15.26:0/3608765724 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76e00042c0 con 0x7f76e407eee0 2024-04-21T10:21:50.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:49.997+0000 7f76eb48b700 1 -- 172.21.15.26:0/3608765724 --> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f76e4104ed0 con 0x7f76d0039660 2024-04-21T10:21:50.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:50.002+0000 7f76da7fc700 1 -- 172.21.15.26:0/3608765724 <== mgr.14120 v2:172.21.15.26:6800/878369551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f76e4104ed0 con 0x7f76d0039660 2024-04-21T10:21:50.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:50.004+0000 7f76eb48b700 1 -- 172.21.15.26:0/3608765724 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f76d0039660 msgr2=0x7f76d003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:50.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:50.004+0000 7f76eb48b700 1 --2- 172.21.15.26:0/3608765724 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f76d0039660 0x7f76d003bb20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f76d4006fd0 tx=0x7f76d4006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:50.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:50.004+0000 7f76eb48b700 1 -- 172.21.15.26:0/3608765724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 msgr2=0x7f76e419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:50.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:50.004+0000 7f76eb48b700 1 --2- 172.21.15.26:0/3608765724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 0x7f76e419e7c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76e00117b0 tx=0x7f76e0011b70 comp rx=0 tx=0).stop 2024-04-21T10:21:50.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:50.004+0000 7f76eb48b700 1 -- 172.21.15.26:0/3608765724 shutdown_connections 2024-04-21T10:21:50.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:50.004+0000 7f76eb48b700 1 --2- 172.21.15.26:0/3608765724 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f76d0039660 0x7f76d003bb20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:50.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:50.004+0000 7f76eb48b700 1 --2- 172.21.15.26:0/3608765724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e407eee0 0x7f76e419e7c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:50.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:50.004+0000 7f76eb48b700 1 -- 172.21.15.26:0/3608765724 >> 172.21.15.26:0/3608765724 conn(0x7f76e4103bc0 msgr2=0x7f76e41047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:50.802 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:50.005+0000 7f76eb48b700 1 -- 172.21.15.26:0/3608765724 shutdown_connections 2024-04-21T10:21:50.802 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:50.005+0000 7f76eb48b700 1 -- 172.21.15.26:0/3608765724 wait complete. 2024-04-21T10:21:51.317 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:51 smithi026 ceph-mon[27965]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:21:51.317 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:51 smithi026 ceph-mon[27965]: Saving service mgr spec with placement count:2 2024-04-21T10:21:51.317 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:51 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' 2024-04-21T10:21:52.340 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.211+0000 7fb9aae43700 1 Processor -- start 2024-04-21T10:21:52.340 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.211+0000 7fb9aae43700 1 -- start start 2024-04-21T10:21:52.340 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.211+0000 7fb9aae43700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 0x7fb9a410b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:52.340 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.211+0000 7fb9aae43700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a407ce70 con 0x7fb9a41090a0 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.211+0000 7fb9a8bdf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 0x7fb9a410b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.211+0000 7fb9a8bdf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 0x7fb9a410b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43030/0 (socket says 172.21.15.26:43030) 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.211+0000 7fb9a8bdf700 1 -- 172.21.15.26:0/304612497 learned_addr learned my addr 172.21.15.26:0/304612497 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.212+0000 7fb9a8bdf700 1 -- 172.21.15.26:0/304612497 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a410b9d0 con 0x7fb9a41090a0 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.212+0000 7fb9a8bdf700 1 --2- 172.21.15.26:0/304612497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 0x7fb9a410b490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb9980098a0 tx=0x7fb998009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4ea74478563014 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.212+0000 7fb9a37fe700 1 -- 172.21.15.26:0/304612497 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb998004160 con 0x7fb9a41090a0 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.212+0000 7fb9a37fe700 1 -- 172.21.15.26:0/304612497 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb998004300 con 0x7fb9a41090a0 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.212+0000 7fb9a37fe700 1 -- 172.21.15.26:0/304612497 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb998011af0 con 0x7fb9a41090a0 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.213+0000 7fb9aae43700 1 -- 172.21.15.26:0/304612497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 msgr2=0x7fb9a410b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.213+0000 7fb9aae43700 1 --2- 172.21.15.26:0/304612497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 0x7fb9a410b490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb9980098a0 tx=0x7fb998009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.213+0000 7fb9aae43700 1 -- 172.21.15.26:0/304612497 shutdown_connections 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.213+0000 7fb9aae43700 1 --2- 172.21.15.26:0/304612497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 0x7fb9a410b490 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.213+0000 7fb9aae43700 1 -- 172.21.15.26:0/304612497 >> 172.21.15.26:0/304612497 conn(0x7fb9a4102af0 msgr2=0x7fb9a4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.213+0000 7fb9aae43700 1 -- 172.21.15.26:0/304612497 shutdown_connections 2024-04-21T10:21:52.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.213+0000 7fb9aae43700 1 -- 172.21.15.26:0/304612497 wait complete. 2024-04-21T10:21:52.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.213+0000 7fb9aae43700 1 Processor -- start 2024-04-21T10:21:52.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.213+0000 7fb9aae43700 1 -- start start 2024-04-21T10:21:52.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.213+0000 7fb9aae43700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 0x7fb9a4108920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:52.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.213+0000 7fb9aae43700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a410c920 con 0x7fb9a41090a0 2024-04-21T10:21:52.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.214+0000 7fb9a8bdf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 0x7fb9a4108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:52.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.214+0000 7fb9a8bdf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 0x7fb9a4108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43040/0 (socket says 172.21.15.26:43040) 2024-04-21T10:21:52.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.214+0000 7fb9a8bdf700 1 -- 172.21.15.26:0/825203609 learned_addr learned my addr 172.21.15.26:0/825203609 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.214+0000 7fb9a8bdf700 1 -- 172.21.15.26:0/825203609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a4108e60 con 0x7fb9a41090a0 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.214+0000 7fb9a8bdf700 1 --2- 172.21.15.26:0/825203609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 0x7fb9a4108920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb998009fb0 tx=0x7fb998011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.214+0000 7fb9a1ffb700 1 -- 172.21.15.26:0/825203609 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9980036a0 con 0x7fb9a41090a0 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.214+0000 7fb9a1ffb700 1 -- 172.21.15.26:0/825203609 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb99801a430 con 0x7fb9a41090a0 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.214+0000 7fb9aae43700 1 -- 172.21.15.26:0/825203609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a4107050 con 0x7fb9a41090a0 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.214+0000 7fb9a1ffb700 1 -- 172.21.15.26:0/825203609 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb998019550 con 0x7fb9a41090a0 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.214+0000 7fb9aae43700 1 -- 172.21.15.26:0/825203609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a4107590 con 0x7fb9a41090a0 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.215+0000 7fb9a1ffb700 1 -- 172.21.15.26:0/825203609 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb998019800 con 0x7fb9a41090a0 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.215+0000 7fb9a1ffb700 1 --2- 172.21.15.26:0/825203609 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fb9940395b0 0x7fb99403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.215+0000 7fb9a1ffb700 1 -- 172.21.15.26:0/825203609 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb99804e8c0 con 0x7fb9a41090a0 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.216+0000 7fb9aae43700 1 -- 172.21.15.26:0/825203609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a40640a0 con 0x7fb9a41090a0 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.216+0000 7fb9a3fff700 1 --2- 172.21.15.26:0/825203609 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fb9940395b0 0x7fb99403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.216+0000 7fb9a3fff700 1 --2- 172.21.15.26:0/825203609 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fb9940395b0 0x7fb99403ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb990006fd0 tx=0x7fb990006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:52.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.221+0000 7fb9a1ffb700 1 -- 172.21.15.26:0/825203609 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb998023c90 con 0x7fb9a41090a0 2024-04-21T10:21:52.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.426+0000 7fb9aae43700 1 -- 172.21.15.26:0/825203609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fb9a4107c90 con 0x7fb9a41090a0 2024-04-21T10:21:52.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.429+0000 7fb9a1ffb700 1 -- 172.21.15.26:0/825203609 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7fb99800bc40 con 0x7fb9a41090a0 2024-04-21T10:21:52.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.431+0000 7fb9aae43700 1 -- 172.21.15.26:0/825203609 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fb9940395b0 msgr2=0x7fb99403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:52.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.431+0000 7fb9aae43700 1 --2- 172.21.15.26:0/825203609 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fb9940395b0 0x7fb99403ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb990006fd0 tx=0x7fb990006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:52.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.431+0000 7fb9aae43700 1 -- 172.21.15.26:0/825203609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 msgr2=0x7fb9a4108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:52.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.431+0000 7fb9aae43700 1 --2- 172.21.15.26:0/825203609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 0x7fb9a4108920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb998009fb0 tx=0x7fb998011940 comp rx=0 tx=0).stop 2024-04-21T10:21:52.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.431+0000 7fb9aae43700 1 -- 172.21.15.26:0/825203609 shutdown_connections 2024-04-21T10:21:52.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.431+0000 7fb9aae43700 1 --2- 172.21.15.26:0/825203609 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fb9940395b0 0x7fb99403ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:52.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.431+0000 7fb9aae43700 1 --2- 172.21.15.26:0/825203609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a41090a0 0x7fb9a4108920 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:52.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.431+0000 7fb9aae43700 1 -- 172.21.15.26:0/825203609 >> 172.21.15.26:0/825203609 conn(0x7fb9a4102af0 msgr2=0x7fb9a4103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:52.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.431+0000 7fb9aae43700 1 -- 172.21.15.26:0/825203609 shutdown_connections 2024-04-21T10:21:52.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:51.431+0000 7fb9aae43700 1 -- 172.21.15.26:0/825203609 wait complete. 2024-04-21T10:21:52.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:52 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/825203609' entity='client.admin' 2024-04-21T10:21:54.049 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:53 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' 2024-04-21T10:21:54.049 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:53 smithi026 ceph-mon[27965]: from='mgr.14120 172.21.15.26:0/1117757637' entity='mgr.x' 2024-04-21T10:21:54.049 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:53 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3777304846' entity='client.admin' 2024-04-21T10:21:54.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.255+0000 7f23d4cd4700 1 Processor -- start 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.255+0000 7f23d4cd4700 1 -- start start 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.255+0000 7f23d4cd4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 0x7f23d0107310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.255+0000 7f23d4cd4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f23d007ce10 con 0x7f23d0106ef0 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.256+0000 7f23ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 0x7f23d0107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.256+0000 7f23ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 0x7f23d0107310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43042/0 (socket says 172.21.15.26:43042) 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.256+0000 7f23ce59c700 1 -- 172.21.15.26:0/1505626702 learned_addr learned my addr 172.21.15.26:0/1505626702 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.256+0000 7f23ce59c700 1 -- 172.21.15.26:0/1505626702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d0107850 con 0x7f23d0106ef0 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.256+0000 7f23ce59c700 1 --2- 172.21.15.26:0/1505626702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 0x7f23d0107310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23c00098a0 tx=0x7f23c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8053d5f3fef1a97 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.256+0000 7f23cd59a700 1 -- 172.21.15.26:0/1505626702 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23c0004160 con 0x7f23d0106ef0 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.256+0000 7f23cd59a700 1 -- 172.21.15.26:0/1505626702 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c0004300 con 0x7f23d0106ef0 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.256+0000 7f23cd59a700 1 -- 172.21.15.26:0/1505626702 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23c0011a20 con 0x7f23d0106ef0 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.257+0000 7f23d4cd4700 1 -- 172.21.15.26:0/1505626702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 msgr2=0x7f23d0107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.257+0000 7f23d4cd4700 1 --2- 172.21.15.26:0/1505626702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 0x7f23d0107310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23c00098a0 tx=0x7f23c0009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.257+0000 7f23d4cd4700 1 -- 172.21.15.26:0/1505626702 shutdown_connections 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.257+0000 7f23d4cd4700 1 --2- 172.21.15.26:0/1505626702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 0x7f23d0107310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.257+0000 7f23d4cd4700 1 -- 172.21.15.26:0/1505626702 >> 172.21.15.26:0/1505626702 conn(0x7f23d0102af0 msgr2=0x7f23d0104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.257+0000 7f23d4cd4700 1 -- 172.21.15.26:0/1505626702 shutdown_connections 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.257+0000 7f23d4cd4700 1 -- 172.21.15.26:0/1505626702 wait complete. 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.258+0000 7f23d4cd4700 1 Processor -- start 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.258+0000 7f23d4cd4700 1 -- start start 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.258+0000 7f23d4cd4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 0x7f23d019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.258+0000 7f23d4cd4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f23d01088c0 con 0x7f23d0106ef0 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.258+0000 7f23ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 0x7f23d019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.259+0000 7f23ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 0x7f23d019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43050/0 (socket says 172.21.15.26:43050) 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.259+0000 7f23ce59c700 1 -- 172.21.15.26:0/3777304846 learned_addr learned my addr 172.21.15.26:0/3777304846 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.259+0000 7f23ce59c700 1 -- 172.21.15.26:0/3777304846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d019ec30 con 0x7f23d0106ef0 2024-04-21T10:21:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.259+0000 7f23ce59c700 1 --2- 172.21.15.26:0/3777304846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 0x7f23d019e6f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f23c00116e0 tx=0x7f23c0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:54.059 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.259+0000 7f23c77fe700 1 -- 172.21.15.26:0/3777304846 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23c0003b20 con 0x7f23d0106ef0 2024-04-21T10:21:54.059 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.259+0000 7f23d4cd4700 1 -- 172.21.15.26:0/3777304846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d019eec0 con 0x7f23d0106ef0 2024-04-21T10:21:54.059 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.259+0000 7f23c77fe700 1 -- 172.21.15.26:0/3777304846 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c001b8f0 con 0x7f23d0106ef0 2024-04-21T10:21:54.059 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.259+0000 7f23c77fe700 1 -- 172.21.15.26:0/3777304846 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23c00238c0 con 0x7f23d0106ef0 2024-04-21T10:21:54.059 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.259+0000 7f23d4cd4700 1 -- 172.21.15.26:0/3777304846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d01a1bb0 con 0x7f23d0106ef0 2024-04-21T10:21:54.059 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.260+0000 7f23d4cd4700 1 -- 172.21.15.26:0/3777304846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d00640a0 con 0x7f23d0106ef0 2024-04-21T10:21:54.059 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.260+0000 7f23c77fe700 1 -- 172.21.15.26:0/3777304846 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f23c0023b70 con 0x7f23d0106ef0 2024-04-21T10:21:54.059 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.261+0000 7f23c77fe700 1 --2- 172.21.15.26:0/3777304846 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f23bc039560 0x7f23bc03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:54.059 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.261+0000 7f23c77fe700 1 -- 172.21.15.26:0/3777304846 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f23c004e050 con 0x7f23d0106ef0 2024-04-21T10:21:54.059 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.261+0000 7f23cdd9b700 1 --2- 172.21.15.26:0/3777304846 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f23bc039560 0x7f23bc03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.261+0000 7f23cdd9b700 1 --2- 172.21.15.26:0/3777304846 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f23bc039560 0x7f23bc03ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f23b8006fd0 tx=0x7f23b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.270+0000 7f23c77fe700 1 -- 172.21.15.26:0/3777304846 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23c001f030 con 0x7f23d0106ef0 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.480+0000 7f23d4cd4700 1 -- 172.21.15.26:0/3777304846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f23d002e260 con 0x7f23d0106ef0 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.482+0000 7f23c77fe700 1 -- 172.21.15.26:0/3777304846 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f23c0018e30 con 0x7f23d0106ef0 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.485+0000 7f23d4cd4700 1 -- 172.21.15.26:0/3777304846 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f23bc039560 msgr2=0x7f23bc03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.485+0000 7f23d4cd4700 1 --2- 172.21.15.26:0/3777304846 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f23bc039560 0x7f23bc03ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f23b8006fd0 tx=0x7f23b8006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.485+0000 7f23d4cd4700 1 -- 172.21.15.26:0/3777304846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 msgr2=0x7f23d019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.485+0000 7f23d4cd4700 1 --2- 172.21.15.26:0/3777304846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 0x7f23d019e6f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f23c00116e0 tx=0x7f23c0011aa0 comp rx=0 tx=0).stop 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.485+0000 7f23d4cd4700 1 -- 172.21.15.26:0/3777304846 shutdown_connections 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.485+0000 7f23d4cd4700 1 --2- 172.21.15.26:0/3777304846 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7f23bc039560 0x7f23bc03ba20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.485+0000 7f23d4cd4700 1 --2- 172.21.15.26:0/3777304846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23d0106ef0 0x7f23d019e6f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.485+0000 7f23d4cd4700 1 -- 172.21.15.26:0/3777304846 >> 172.21.15.26:0/3777304846 conn(0x7f23d0102af0 msgr2=0x7f23d0195360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.485+0000 7f23d4cd4700 1 -- 172.21.15.26:0/3777304846 shutdown_connections 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:53.485+0000 7f23d4cd4700 1 -- 172.21.15.26:0/3777304846 wait complete. 2024-04-21T10:21:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling the dashboard module... 2024-04-21T10:21:55.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:54 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1835694144' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-21T10:21:55.185 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:54 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: ignoring --setuser ceph since I am not root 2024-04-21T10:21:55.185 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:54 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: ignoring --setgroup ceph since I am not root 2024-04-21T10:21:55.186 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:55.049+0000 7f198f2a5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T10:21:55.574 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:55.243+0000 7f198f2a5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:21:55.574 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:55.350+0000 7f198f2a5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T10:21:55.577 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.517+0000 7fa196a48700 1 Processor -- start 2024-04-21T10:21:55.583 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.517+0000 7fa196a48700 1 -- start start 2024-04-21T10:21:55.583 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.517+0000 7fa196a48700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 0x7fa19010b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:55.583 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.517+0000 7fa196a48700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa190063d40 con 0x7fa1901090d0 2024-04-21T10:21:55.583 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.517+0000 7fa18ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 0x7fa19010b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:55.583 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.518+0000 7fa18ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 0x7fa19010b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43062/0 (socket says 172.21.15.26:43062) 2024-04-21T10:21:55.583 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.518+0000 7fa18ffff700 1 -- 172.21.15.26:0/783230380 learned_addr learned my addr 172.21.15.26:0/783230380 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:55.583 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.518+0000 7fa18ffff700 1 -- 172.21.15.26:0/783230380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19010ba00 con 0x7fa1901090d0 2024-04-21T10:21:55.583 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.518+0000 7fa18ffff700 1 --2- 172.21.15.26:0/783230380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 0x7fa19010b4c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa188009e80 tx=0x7fa18800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16600b259d9fc32 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:55.583 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.518+0000 7fa18effd700 1 -- 172.21.15.26:0/783230380 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa188017070 con 0x7fa1901090d0 2024-04-21T10:21:55.583 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.518+0000 7fa18effd700 1 -- 172.21.15.26:0/783230380 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18800b8c0 con 0x7fa1901090d0 2024-04-21T10:21:55.583 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.518+0000 7fa18effd700 1 -- 172.21.15.26:0/783230380 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa188004490 con 0x7fa1901090d0 2024-04-21T10:21:55.583 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.519+0000 7fa196a48700 1 -- 172.21.15.26:0/783230380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 msgr2=0x7fa19010b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:55.584 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.519+0000 7fa196a48700 1 --2- 172.21.15.26:0/783230380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 0x7fa19010b4c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa188009e80 tx=0x7fa18800b330 comp rx=0 tx=0).stop 2024-04-21T10:21:55.584 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.519+0000 7fa196a48700 1 -- 172.21.15.26:0/783230380 shutdown_connections 2024-04-21T10:21:55.584 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.519+0000 7fa196a48700 1 --2- 172.21.15.26:0/783230380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 0x7fa19010b4c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:55.584 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.519+0000 7fa196a48700 1 -- 172.21.15.26:0/783230380 >> 172.21.15.26:0/783230380 conn(0x7fa190102b80 msgr2=0x7fa190104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:55.584 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.519+0000 7fa196a48700 1 -- 172.21.15.26:0/783230380 shutdown_connections 2024-04-21T10:21:55.584 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.519+0000 7fa196a48700 1 -- 172.21.15.26:0/783230380 wait complete. 2024-04-21T10:21:55.584 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.520+0000 7fa196a48700 1 Processor -- start 2024-04-21T10:21:55.584 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.520+0000 7fa196a48700 1 -- start start 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.521+0000 7fa196a48700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 0x7fa19019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.521+0000 7fa196a48700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa190107750 con 0x7fa1901090d0 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.521+0000 7fa18ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 0x7fa19019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.521+0000 7fa18ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 0x7fa19019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43074/0 (socket says 172.21.15.26:43074) 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.521+0000 7fa18ffff700 1 -- 172.21.15.26:0/1835694144 learned_addr learned my addr 172.21.15.26:0/1835694144 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.521+0000 7fa18ffff700 1 -- 172.21.15.26:0/1835694144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19019a7a0 con 0x7fa1901090d0 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.522+0000 7fa18ffff700 1 --2- 172.21.15.26:0/1835694144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 0x7fa19019a260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa188009e50 tx=0x7fa18800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.522+0000 7fa18d7fa700 1 -- 172.21.15.26:0/1835694144 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa188020020 con 0x7fa1901090d0 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.522+0000 7fa18d7fa700 1 -- 172.21.15.26:0/1835694144 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa188003b50 con 0x7fa1901090d0 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.522+0000 7fa18d7fa700 1 -- 172.21.15.26:0/1835694144 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa188017040 con 0x7fa1901090d0 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.522+0000 7fa196a48700 1 -- 172.21.15.26:0/1835694144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19019aa30 con 0x7fa1901090d0 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.522+0000 7fa196a48700 1 -- 172.21.15.26:0/1835694144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa19019d720 con 0x7fa1901090d0 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.523+0000 7fa18d7fa700 1 -- 172.21.15.26:0/1835694144 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa188023020 con 0x7fa1901090d0 2024-04-21T10:21:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.523+0000 7fa18d7fa700 1 --2- 172.21.15.26:0/1835694144 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fa17c03db50 0x7fa17c040010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.523+0000 7fa18d7fa700 1 -- 172.21.15.26:0/1835694144 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa18800f050 con 0x7fa1901090d0 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.523+0000 7fa18f7fe700 1 --2- 172.21.15.26:0/1835694144 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fa17c03db50 0x7fa17c040010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.524+0000 7fa18f7fe700 1 --2- 172.21.15.26:0/1835694144 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fa17c03db50 0x7fa17c040010 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa178006fd0 tx=0x7fa178006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.524+0000 7fa196a48700 1 -- 172.21.15.26:0/1835694144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1900640a0 con 0x7fa1901090d0 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.529+0000 7fa18d7fa700 1 -- 172.21.15.26:0/1835694144 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa18802a020 con 0x7fa1901090d0 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.801+0000 7fa196a48700 1 -- 172.21.15.26:0/1835694144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa19002e100 con 0x7fa1901090d0 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.852+0000 7fa18d7fa700 1 -- 172.21.15.26:0/1835694144 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa18802d350 con 0x7fa1901090d0 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.854+0000 7fa18d7fa700 1 -- 172.21.15.26:0/1835694144 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fa18801a2d0 con 0x7fa1901090d0 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.855+0000 7fa196a48700 1 -- 172.21.15.26:0/1835694144 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fa17c03db50 msgr2=0x7fa17c040010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.855+0000 7fa196a48700 1 --2- 172.21.15.26:0/1835694144 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fa17c03db50 0x7fa17c040010 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa178006fd0 tx=0x7fa178006e40 comp rx=0 tx=0).stop 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.855+0000 7fa196a48700 1 -- 172.21.15.26:0/1835694144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 msgr2=0x7fa19019a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.855+0000 7fa196a48700 1 --2- 172.21.15.26:0/1835694144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 0x7fa19019a260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa188009e50 tx=0x7fa18800bd50 comp rx=0 tx=0).stop 2024-04-21T10:21:55.586 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.855+0000 7fa196a48700 1 -- 172.21.15.26:0/1835694144 shutdown_connections 2024-04-21T10:21:55.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.855+0000 7fa196a48700 1 --2- 172.21.15.26:0/1835694144 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fa17c03db50 0x7fa17c040010 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:55.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.855+0000 7fa196a48700 1 --2- 172.21.15.26:0/1835694144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1901090d0 0x7fa19019a260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:55.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.855+0000 7fa196a48700 1 -- 172.21.15.26:0/1835694144 >> 172.21.15.26:0/1835694144 conn(0x7fa190102b80 msgr2=0x7fa190190e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:55.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.855+0000 7fa196a48700 1 -- 172.21.15.26:0/1835694144 shutdown_connections 2024-04-21T10:21:55.587 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:54.855+0000 7fa196a48700 1 -- 172.21.15.26:0/1835694144 wait complete. 2024-04-21T10:21:55.856 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:55.683+0000 7f198f2a5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T10:21:55.857 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:55.772+0000 7f198f2a5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T10:21:56.142 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:55 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1835694144' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-21T10:21:56.142 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:55 smithi026 ceph-mon[27965]: mgrmap e9: x(active, since 18s) 2024-04-21T10:21:57.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:21:56 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/207538816' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-21T10:21:57.185 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:57 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:57.048+0000 7f198f2a5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T10:21:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-04-21T10:21:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-21T10:21:57.253 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T10:21:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-21T10:21:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-21T10:21:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-04-21T10:21:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.368+0000 7fbe99686700 1 Processor -- start 2024-04-21T10:21:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.368+0000 7fbe99686700 1 -- start start 2024-04-21T10:21:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.368+0000 7fbe99686700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 0x7fbe941072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.368+0000 7fbe99686700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9407ce30 con 0x7fbe94106ed0 2024-04-21T10:21:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.368+0000 7fbe92ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 0x7fbe941072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:57.255 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.368+0000 7fbe92ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 0x7fbe941072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59880/0 (socket says 172.21.15.26:59880) 2024-04-21T10:21:57.255 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.368+0000 7fbe92ffd700 1 -- 172.21.15.26:0/1319640977 learned_addr learned my addr 172.21.15.26:0/1319640977 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:57.255 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.369+0000 7fbe92ffd700 1 -- 172.21.15.26:0/1319640977 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe94107830 con 0x7fbe94106ed0 2024-04-21T10:21:57.255 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.369+0000 7fbe92ffd700 1 --2- 172.21.15.26:0/1319640977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 0x7fbe941072f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe840098a0 tx=0x7fbe84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a109ef30a25ae39 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:57.255 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.369+0000 7fbe91ffb700 1 -- 172.21.15.26:0/1319640977 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe84004160 con 0x7fbe94106ed0 2024-04-21T10:21:57.255 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.369+0000 7fbe91ffb700 1 -- 172.21.15.26:0/1319640977 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe84004300 con 0x7fbe94106ed0 2024-04-21T10:21:57.255 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.369+0000 7fbe91ffb700 1 -- 172.21.15.26:0/1319640977 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe84011a20 con 0x7fbe94106ed0 2024-04-21T10:21:57.255 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.370+0000 7fbe99686700 1 -- 172.21.15.26:0/1319640977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 msgr2=0x7fbe941072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.370+0000 7fbe99686700 1 --2- 172.21.15.26:0/1319640977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 0x7fbe941072f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe840098a0 tx=0x7fbe84009bb0 comp rx=0 tx=0).stop 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.370+0000 7fbe99686700 1 -- 172.21.15.26:0/1319640977 shutdown_connections 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.370+0000 7fbe99686700 1 --2- 172.21.15.26:0/1319640977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 0x7fbe941072f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.370+0000 7fbe99686700 1 -- 172.21.15.26:0/1319640977 >> 172.21.15.26:0/1319640977 conn(0x7fbe94102af0 msgr2=0x7fbe94104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.370+0000 7fbe99686700 1 -- 172.21.15.26:0/1319640977 shutdown_connections 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.370+0000 7fbe99686700 1 -- 172.21.15.26:0/1319640977 wait complete. 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.370+0000 7fbe99686700 1 Processor -- start 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.370+0000 7fbe99686700 1 -- start start 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.371+0000 7fbe99686700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 0x7fbe9419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.371+0000 7fbe99686700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe941088a0 con 0x7fbe94106ed0 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.371+0000 7fbe92ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 0x7fbe9419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.371+0000 7fbe92ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 0x7fbe9419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59886/0 (socket says 172.21.15.26:59886) 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.371+0000 7fbe92ffd700 1 -- 172.21.15.26:0/207538816 learned_addr learned my addr 172.21.15.26:0/207538816 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:21:57.256 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.371+0000 7fbe92ffd700 1 -- 172.21.15.26:0/207538816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9419ec20 con 0x7fbe94106ed0 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.371+0000 7fbe92ffd700 1 --2- 172.21.15.26:0/207538816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 0x7fbe9419e6e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe840116e0 tx=0x7fbe84011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.371+0000 7fbe7bfff700 1 -- 172.21.15.26:0/207538816 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe84003b20 con 0x7fbe94106ed0 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.371+0000 7fbe7bfff700 1 -- 172.21.15.26:0/207538816 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe8401b8f0 con 0x7fbe94106ed0 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.372+0000 7fbe99686700 1 -- 172.21.15.26:0/207538816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9419eeb0 con 0x7fbe94106ed0 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.372+0000 7fbe7bfff700 1 -- 172.21.15.26:0/207538816 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe840238c0 con 0x7fbe94106ed0 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.372+0000 7fbe99686700 1 -- 172.21.15.26:0/207538816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe941a1ba0 con 0x7fbe94106ed0 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.373+0000 7fbe7bfff700 1 -- 172.21.15.26:0/207538816 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fbe84023b40 con 0x7fbe94106ed0 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.373+0000 7fbe99686700 1 -- 172.21.15.26:0/207538816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe940640a0 con 0x7fbe94106ed0 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.373+0000 7fbe7bfff700 1 --2- 172.21.15.26:0/207538816 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fbe7c039610 0x7fbe7c03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.373+0000 7fbe927fc700 1 -- 172.21.15.26:0/207538816 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fbe7c039610 msgr2=0x7fbe7c03bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/878369551 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.373+0000 7fbe7bfff700 1 -- 172.21.15.26:0/207538816 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbe8404e020 con 0x7fbe94106ed0 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.373+0000 7fbe927fc700 1 --2- 172.21.15.26:0/207538816 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fbe7c039610 0x7fbe7c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T10:21:57.257 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.378+0000 7fbe7bfff700 1 -- 172.21.15.26:0/207538816 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe8401f030 con 0x7fbe94106ed0 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.574+0000 7fbe927fc700 1 -- 172.21.15.26:0/207538816 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fbe7c039610 msgr2=0x7fbe7c03bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/878369551 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.574+0000 7fbe927fc700 1 --2- 172.21.15.26:0/207538816 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fbe7c039610 0x7fbe7c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.646+0000 7fbe99686700 1 -- 172.21.15.26:0/207538816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fbe9402e1d0 con 0x7fbe94106ed0 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.647+0000 7fbe7bfff700 1 -- 172.21.15.26:0/207538816 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fbe84010380 con 0x7fbe94106ed0 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.649+0000 7fbe99686700 1 -- 172.21.15.26:0/207538816 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fbe7c039610 msgr2=0x7fbe7c03bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.649+0000 7fbe99686700 1 --2- 172.21.15.26:0/207538816 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fbe7c039610 0x7fbe7c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.649+0000 7fbe99686700 1 -- 172.21.15.26:0/207538816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 msgr2=0x7fbe9419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.649+0000 7fbe99686700 1 --2- 172.21.15.26:0/207538816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 0x7fbe9419e6e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe840116e0 tx=0x7fbe84011aa0 comp rx=0 tx=0).stop 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.649+0000 7fbe99686700 1 -- 172.21.15.26:0/207538816 shutdown_connections 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.649+0000 7fbe99686700 1 --2- 172.21.15.26:0/207538816 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7fbe7c039610 0x7fbe7c03bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.649+0000 7fbe99686700 1 --2- 172.21.15.26:0/207538816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106ed0 0x7fbe9419e6e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.649+0000 7fbe99686700 1 -- 172.21.15.26:0/207538816 >> 172.21.15.26:0/207538816 conn(0x7fbe94102af0 msgr2=0x7fbe94195340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.649+0000 7fbe99686700 1 -- 172.21.15.26:0/207538816 shutdown_connections 2024-04-21T10:21:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:56.649+0000 7fbe99686700 1 -- 172.21.15.26:0/207538816 wait complete. 2024-04-21T10:21:57.259 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for the mgr to restart... 2024-04-21T10:21:57.259 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr epoch 9... 2024-04-21T10:21:57.480 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:57 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:57.227+0000 7f198f2a5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T10:21:57.481 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:57 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:57.409+0000 7f198f2a5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T10:21:57.819 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:57 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:57.723+0000 7f198f2a5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T10:21:57.819 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:57 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:57.819+0000 7f198f2a5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T10:21:58.185 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:58.031+0000 7f198f2a5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T10:21:58.685 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:58.233+0000 7f198f2a5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:21:59.185 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:58.863+0000 7f198f2a5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T10:21:59.617 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:59 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:59.304+0000 7f198f2a5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T10:21:59.617 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:59 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:59.405+0000 7f198f2a5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T10:21:59.617 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:21:59 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:21:59.617+0000 7f198f2a5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T10:22:00.435 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:22:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:22:00.166+0000 7f198f2a5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T10:22:00.435 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:22:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:22:00.262+0000 7f198f2a5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T10:22:00.880 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:22:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:22:00.518+0000 7f198f2a5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T10:22:00.880 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:22:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:22:00.659+0000 7f198f2a5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T10:22:00.880 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:22:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:22:00.758+0000 7f198f2a5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T10:22:00.880 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:22:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:22:00.880+0000 7f198f2a5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T10:22:01.685 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:22:01 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:22:01.218+0000 7f198f2a5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T10:22:02.133 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:22:01 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:22:01.851+0000 7f198f2a5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T10:22:02.134 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:22:02 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:22:02.080+0000 7f198f2a5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T10:22:02.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:02 smithi026 ceph-mon[27965]: Active manager daemon x restarted 2024-04-21T10:22:02.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:02 smithi026 ceph-mon[27965]: Activating manager daemon x 2024-04-21T10:22:02.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:02 smithi026 ceph-mon[27965]: osdmap e3: 0 total, 0 up, 0 in 2024-04-21T10:22:02.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:02 smithi026 ceph-mon[27965]: mgrmap e10: x(active, starting, since 0.00350118s) 2024-04-21T10:22:02.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:22:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T10:22:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T10:22:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T10:22:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T10:22:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:02 smithi026 ceph-mon[27965]: Manager daemon x is now available 2024-04-21T10:22:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:22:03.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T10:22:03.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T10:22:03.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:03 smithi026 ceph-mon[27965]: mgrmap e11: x(active, since 1.00633s) 2024-04-21T10:22:04.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2024-04-21T10:22:04.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-21T10:22:04.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-21T10:22:04.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2024-04-21T10:22:04.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.679+0000 7ffa43bea700 1 Processor -- start 2024-04-21T10:22:04.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.680+0000 7ffa43bea700 1 -- start start 2024-04-21T10:22:04.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.680+0000 7ffa43bea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 0x7ffa3c10b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:04.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.680+0000 7ffa43bea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c063ca0 con 0x7ffa3c109170 2024-04-21T10:22:04.051 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.680+0000 7ffa41986700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 0x7ffa3c10b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.680+0000 7ffa41986700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 0x7ffa3c10b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59892/0 (socket says 172.21.15.26:59892) 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.680+0000 7ffa41986700 1 -- 172.21.15.26:0/686336416 learned_addr learned my addr 172.21.15.26:0/686336416 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.680+0000 7ffa41986700 1 -- 172.21.15.26:0/686336416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c10baa0 con 0x7ffa3c109170 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.681+0000 7ffa41986700 1 --2- 172.21.15.26:0/686336416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 0x7ffa3c10b560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa380098a0 tx=0x7ffa38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc231ac3d8dbdd6c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.681+0000 7ffa40984700 1 -- 172.21.15.26:0/686336416 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa38004160 con 0x7ffa3c109170 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.681+0000 7ffa40984700 1 -- 172.21.15.26:0/686336416 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa38004300 con 0x7ffa3c109170 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.681+0000 7ffa40984700 1 -- 172.21.15.26:0/686336416 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa38011a20 con 0x7ffa3c109170 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.682+0000 7ffa43bea700 1 -- 172.21.15.26:0/686336416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 msgr2=0x7ffa3c10b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.682+0000 7ffa43bea700 1 --2- 172.21.15.26:0/686336416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 0x7ffa3c10b560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa380098a0 tx=0x7ffa38009bb0 comp rx=0 tx=0).stop 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.682+0000 7ffa43bea700 1 -- 172.21.15.26:0/686336416 shutdown_connections 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.682+0000 7ffa43bea700 1 --2- 172.21.15.26:0/686336416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 0x7ffa3c10b560 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.682+0000 7ffa43bea700 1 -- 172.21.15.26:0/686336416 >> 172.21.15.26:0/686336416 conn(0x7ffa3c102b80 msgr2=0x7ffa3c104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.682+0000 7ffa43bea700 1 -- 172.21.15.26:0/686336416 shutdown_connections 2024-04-21T10:22:04.052 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.682+0000 7ffa43bea700 1 -- 172.21.15.26:0/686336416 wait complete. 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.682+0000 7ffa43bea700 1 Processor -- start 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.682+0000 7ffa43bea700 1 -- start start 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.683+0000 7ffa43bea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 0x7ffa3c1089f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.683+0000 7ffa43bea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c10c9f0 con 0x7ffa3c109170 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.683+0000 7ffa41986700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 0x7ffa3c1089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.683+0000 7ffa41986700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 0x7ffa3c1089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59896/0 (socket says 172.21.15.26:59896) 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.683+0000 7ffa41986700 1 -- 172.21.15.26:0/2555099969 learned_addr learned my addr 172.21.15.26:0/2555099969 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.683+0000 7ffa41986700 1 -- 172.21.15.26:0/2555099969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c108f30 con 0x7ffa3c109170 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.683+0000 7ffa41986700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 0x7ffa3c1089f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffa380116e0 tx=0x7ffa38011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.683+0000 7ffa32ffd700 1 -- 172.21.15.26:0/2555099969 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa38003b20 con 0x7ffa3c109170 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.683+0000 7ffa32ffd700 1 -- 172.21.15.26:0/2555099969 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa3801b8f0 con 0x7ffa3c109170 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.683+0000 7ffa32ffd700 1 -- 172.21.15.26:0/2555099969 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa380238c0 con 0x7ffa3c109170 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.683+0000 7ffa43bea700 1 -- 172.21.15.26:0/2555099969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c107120 con 0x7ffa3c109170 2024-04-21T10:22:04.053 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.684+0000 7ffa43bea700 1 -- 172.21.15.26:0/2555099969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c107660 con 0x7ffa3c109170 2024-04-21T10:22:04.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.684+0000 7ffa32ffd700 1 -- 172.21.15.26:0/2555099969 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7ffa38023b70 con 0x7ffa3c109170 2024-04-21T10:22:04.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.684+0000 7ffa32ffd700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 0x7ffa2803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:04.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.684+0000 7ffa32ffd700 1 -- 172.21.15.26:0/2555099969 --> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffa2803c180 con 0x7ffa280395b0 2024-04-21T10:22:04.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.685+0000 7ffa32ffd700 1 -- 172.21.15.26:0/2555099969 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffa3804ef20 con 0x7ffa3c109170 2024-04-21T10:22:04.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.685+0000 7ffa41185700 1 -- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 msgr2=0x7ffa2803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/878369551 2024-04-21T10:22:04.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.685+0000 7ffa41185700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 0x7ffa2803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T10:22:04.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.886+0000 7ffa41185700 1 -- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 msgr2=0x7ffa2803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/878369551 2024-04-21T10:22:04.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:57.886+0000 7ffa41185700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 0x7ffa2803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T10:22:04.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:58.287+0000 7ffa41185700 1 -- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 msgr2=0x7ffa2803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/878369551 2024-04-21T10:22:04.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:58.287+0000 7ffa41185700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 0x7ffa2803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-21T10:22:04.054 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:59.088+0000 7ffa41185700 1 -- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 msgr2=0x7ffa2803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/878369551 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:21:59.088+0000 7ffa41185700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 0x7ffa2803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:00.689+0000 7ffa41185700 1 -- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 msgr2=0x7ffa2803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/878369551 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:00.689+0000 7ffa41185700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 0x7ffa2803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:02.084+0000 7ffa32ffd700 1 -- 172.21.15.26:0/2555099969 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7ffa3801baa0 con 0x7ffa3c109170 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:02.084+0000 7ffa32ffd700 1 -- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 msgr2=0x7ffa2803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:02.084+0000 7ffa32ffd700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 0x7ffa2803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.086+0000 7ffa32ffd700 1 -- 172.21.15.26:0/2555099969 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7ffa3802c770 con 0x7ffa3c109170 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.086+0000 7ffa32ffd700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ffa2803cfe0 0x7ffa2803f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.086+0000 7ffa32ffd700 1 -- 172.21.15.26:0/2555099969 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffa2803c180 con 0x7ffa2803cfe0 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.087+0000 7ffa41185700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ffa2803cfe0 0x7ffa2803f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.087+0000 7ffa41185700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ffa2803cfe0 0x7ffa2803f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c003a10 tx=0x7ffa2c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.088+0000 7ffa32ffd700 1 -- 172.21.15.26:0/2555099969 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ffa2803c180 con 0x7ffa2803cfe0 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa43bea700 1 -- 172.21.15.26:0/2555099969 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ffa3c0640a0 con 0x7ffa2803cfe0 2024-04-21T10:22:04.055 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa32ffd700 1 -- 172.21.15.26:0/2555099969 <== mgr.14152 v2:172.21.15.26:6800/72610606 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ffa3c0640a0 con 0x7ffa2803cfe0 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa43bea700 1 -- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ffa2803cfe0 msgr2=0x7ffa2803f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa43bea700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ffa2803cfe0 0x7ffa2803f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c003a10 tx=0x7ffa2c0092b0 comp rx=0 tx=0).stop 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa43bea700 1 -- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 msgr2=0x7ffa3c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa43bea700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 0x7ffa3c1089f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffa380116e0 tx=0x7ffa38011aa0 comp rx=0 tx=0).stop 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa43bea700 1 -- 172.21.15.26:0/2555099969 shutdown_connections 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa43bea700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ffa2803cfe0 0x7ffa2803f3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa43bea700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa3c109170 0x7ffa3c1089f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa43bea700 1 --2- 172.21.15.26:0/2555099969 >> [v2:172.21.15.26:6800/878369551,v1:172.21.15.26:6801/878369551] conn(0x7ffa280395b0 0x7ffa2803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa43bea700 1 -- 172.21.15.26:0/2555099969 >> 172.21.15.26:0/2555099969 conn(0x7ffa3c102b80 msgr2=0x7ffa3c103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa43bea700 1 -- 172.21.15.26:0/2555099969 shutdown_connections 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:03.095+0000 7ffa43bea700 1 -- 172.21.15.26:0/2555099969 wait complete. 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:mgr epoch 9 is available 2024-04-21T10:22:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:Generating a dashboard self-signed certificate... 2024-04-21T10:22:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:04 smithi026 ceph-mon[27965]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-21T10:22:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:04 smithi026 ceph-mon[27965]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-21T10:22:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:04 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.551+0000 7f0269af1700 1 Processor -- start 2024-04-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.551+0000 7f0269af1700 1 -- start start 2024-04-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.551+0000 7f0269af1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 0x7f0264108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.551+0000 7f0269af1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0264109000 con 0x7f0264108610 2024-04-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.552+0000 7f02637fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 0x7f0264108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.552+0000 7f02637fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 0x7f0264108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59956/0 (socket says 172.21.15.26:59956) 2024-04-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.552+0000 7f02637fe700 1 -- 172.21.15.26:0/3212268600 learned_addr learned my addr 172.21.15.26:0/3212268600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.552+0000 7f02637fe700 1 -- 172.21.15.26:0/3212268600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0264109810 con 0x7f0264108610 2024-04-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.552+0000 7f02637fe700 1 --2- 172.21.15.26:0/3212268600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 0x7f0264108a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02540098a0 tx=0x7f0254009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244854067fe845e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:05.297 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.552+0000 7f02627fc700 1 -- 172.21.15.26:0/3212268600 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0254004160 con 0x7f0264108610 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.552+0000 7f02627fc700 1 -- 172.21.15.26:0/3212268600 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0254004300 con 0x7f0264108610 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.552+0000 7f02627fc700 1 -- 172.21.15.26:0/3212268600 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0254011a20 con 0x7f0264108610 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.553+0000 7f0269af1700 1 -- 172.21.15.26:0/3212268600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 msgr2=0x7f0264108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.553+0000 7f0269af1700 1 --2- 172.21.15.26:0/3212268600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 0x7f0264108a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02540098a0 tx=0x7f0254009bb0 comp rx=0 tx=0).stop 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.553+0000 7f0269af1700 1 -- 172.21.15.26:0/3212268600 shutdown_connections 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.553+0000 7f0269af1700 1 --2- 172.21.15.26:0/3212268600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 0x7f0264108a30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.553+0000 7f0269af1700 1 -- 172.21.15.26:0/3212268600 >> 172.21.15.26:0/3212268600 conn(0x7f0264103c70 msgr2=0x7f0264106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.553+0000 7f0269af1700 1 -- 172.21.15.26:0/3212268600 shutdown_connections 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.553+0000 7f0269af1700 1 -- 172.21.15.26:0/3212268600 wait complete. 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.554+0000 7f0269af1700 1 Processor -- start 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.554+0000 7f0269af1700 1 -- start start 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.554+0000 7f0269af1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 0x7f026419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.554+0000 7f0269af1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f026410cbc0 con 0x7f0264108610 2024-04-21T10:22:05.298 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.554+0000 7f02637fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 0x7f026419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.554+0000 7f02637fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 0x7f026419c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59964/0 (socket says 172.21.15.26:59964) 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.554+0000 7f02637fe700 1 -- 172.21.15.26:0/2741619752 learned_addr learned my addr 172.21.15.26:0/2741619752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.555+0000 7f02637fe700 1 -- 172.21.15.26:0/2741619752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026419c9f0 con 0x7f0264108610 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.555+0000 7f02637fe700 1 --2- 172.21.15.26:0/2741619752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 0x7f026419c4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0254000c00 tx=0x7f0254011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.555+0000 7f0260ff9700 1 -- 172.21.15.26:0/2741619752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0254003aa0 con 0x7f0264108610 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.555+0000 7f0269af1700 1 -- 172.21.15.26:0/2741619752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026419cc80 con 0x7f0264108610 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.555+0000 7f0260ff9700 1 -- 172.21.15.26:0/2741619752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f025401b8f0 con 0x7f0264108610 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.555+0000 7f0260ff9700 1 -- 172.21.15.26:0/2741619752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0254023890 con 0x7f0264108610 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.555+0000 7f0269af1700 1 -- 172.21.15.26:0/2741619752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f026419f970 con 0x7f0264108610 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.556+0000 7f0260ff9700 1 -- 172.21.15.26:0/2741619752 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f025401b440 con 0x7f0264108610 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.556+0000 7f0269af1700 1 -- 172.21.15.26:0/2741619752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f026410cd40 con 0x7f0264108610 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.556+0000 7f0260ff9700 1 --2- 172.21.15.26:0/2741619752 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f024c0394e0 0x7f024c03b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.556+0000 7f0260ff9700 1 -- 172.21.15.26:0/2741619752 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f025404dda0 con 0x7f0264108610 2024-04-21T10:22:05.299 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.557+0000 7f0262ffd700 1 --2- 172.21.15.26:0/2741619752 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f024c0394e0 0x7f024c03b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.557+0000 7f0262ffd700 1 --2- 172.21.15.26:0/2741619752 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f024c0394e0 0x7f024c03b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0258006fd0 tx=0x7f0258006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.562+0000 7f0260ff9700 1 -- 172.21.15.26:0/2741619752 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f025401f030 con 0x7f0264108610 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.695+0000 7f0260ff9700 1 -- 172.21.15.26:0/2741619752 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f025402c770 con 0x7f0264108610 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.780+0000 7f0269af1700 1 -- 172.21.15.26:0/2741619752 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0264104e80 con 0x7f024c0394e0 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.824+0000 7f0260ff9700 1 -- 172.21.15.26:0/2741619752 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0264104e80 con 0x7f024c0394e0 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.826+0000 7f0269af1700 1 -- 172.21.15.26:0/2741619752 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f024c0394e0 msgr2=0x7f024c03b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.826+0000 7f0269af1700 1 --2- 172.21.15.26:0/2741619752 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f024c0394e0 0x7f024c03b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0258006fd0 tx=0x7f0258006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.826+0000 7f0269af1700 1 -- 172.21.15.26:0/2741619752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 msgr2=0x7f026419c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.826+0000 7f0269af1700 1 --2- 172.21.15.26:0/2741619752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 0x7f026419c4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0254000c00 tx=0x7f0254011aa0 comp rx=0 tx=0).stop 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.826+0000 7f0269af1700 1 -- 172.21.15.26:0/2741619752 shutdown_connections 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.826+0000 7f0269af1700 1 --2- 172.21.15.26:0/2741619752 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f024c0394e0 0x7f024c03b9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.826+0000 7f0269af1700 1 --2- 172.21.15.26:0/2741619752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0264108610 0x7f026419c4b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.826+0000 7f0269af1700 1 -- 172.21.15.26:0/2741619752 >> 172.21.15.26:0/2741619752 conn(0x7f0264103c70 msgr2=0x7f0264104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:05.300 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.826+0000 7f0269af1700 1 -- 172.21.15.26:0/2741619752 shutdown_connections 2024-04-21T10:22:05.301 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:04.826+0000 7f0269af1700 1 -- 172.21.15.26:0/2741619752 wait complete. 2024-04-21T10:22:05.301 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial admin user... 2024-04-21T10:22:05.989 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:05 smithi026 ceph-mon[27965]: [21/Apr/2024:10:22:04] ENGINE Bus STARTING 2024-04-21T10:22:05.989 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:05 smithi026 ceph-mon[27965]: [21/Apr/2024:10:22:04] ENGINE Serving on https://172.21.15.26:7150 2024-04-21T10:22:05.989 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:05 smithi026 ceph-mon[27965]: [21/Apr/2024:10:22:04] ENGINE Serving on http://172.21.15.26:8765 2024-04-21T10:22:05.989 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:05 smithi026 ceph-mon[27965]: [21/Apr/2024:10:22:04] ENGINE Bus STARTED 2024-04-21T10:22:05.989 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:05 smithi026 ceph-mon[27965]: mgrmap e12: x(active, since 2s) 2024-04-21T10:22:05.989 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:05 smithi026 ceph-mon[27965]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:22:05.989 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:05.989 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:05.989 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:07.309 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$8gI5VFdZvZ1.Q7pxzYkjauY3p1SCl19NjW1J0JLAjHY4U1woE9vFy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713694926, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.741+0000 7ff023ed7700 1 Processor -- start 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.741+0000 7ff023ed7700 1 -- start start 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.741+0000 7ff023ed7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 0x7ff01c106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.741+0000 7ff023ed7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c107330 con 0x7ff01c106940 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.742+0000 7ff021c73700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 0x7ff01c106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.742+0000 7ff021c73700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 0x7ff01c106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45470/0 (socket says 172.21.15.26:45470) 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.742+0000 7ff021c73700 1 -- 172.21.15.26:0/1694094787 learned_addr learned my addr 172.21.15.26:0/1694094787 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.742+0000 7ff021c73700 1 -- 172.21.15.26:0/1694094787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c109970 con 0x7ff01c106940 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.742+0000 7ff021c73700 1 --2- 172.21.15.26:0/1694094787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 0x7ff01c106d60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff0180098a0 tx=0x7ff018009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c560ab738923f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.742+0000 7ff020c71700 1 -- 172.21.15.26:0/1694094787 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff018004160 con 0x7ff01c106940 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.742+0000 7ff020c71700 1 -- 172.21.15.26:0/1694094787 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff018004300 con 0x7ff01c106940 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.742+0000 7ff020c71700 1 -- 172.21.15.26:0/1694094787 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff018011a20 con 0x7ff01c106940 2024-04-21T10:22:07.310 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.743+0000 7ff023ed7700 1 -- 172.21.15.26:0/1694094787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 msgr2=0x7ff01c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.743+0000 7ff023ed7700 1 --2- 172.21.15.26:0/1694094787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 0x7ff01c106d60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff0180098a0 tx=0x7ff018009bb0 comp rx=0 tx=0).stop 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.744+0000 7ff023ed7700 1 -- 172.21.15.26:0/1694094787 shutdown_connections 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.744+0000 7ff023ed7700 1 --2- 172.21.15.26:0/1694094787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 0x7ff01c106d60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.744+0000 7ff023ed7700 1 -- 172.21.15.26:0/1694094787 >> 172.21.15.26:0/1694094787 conn(0x7ff01c1022b0 msgr2=0x7ff01c1046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.744+0000 7ff023ed7700 1 -- 172.21.15.26:0/1694094787 shutdown_connections 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.744+0000 7ff023ed7700 1 -- 172.21.15.26:0/1694094787 wait complete. 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.744+0000 7ff023ed7700 1 Processor -- start 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.744+0000 7ff023ed7700 1 -- start start 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.745+0000 7ff023ed7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 0x7ff01c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.745+0000 7ff023ed7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c10cba0 con 0x7ff01c106940 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.745+0000 7ff021c73700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 0x7ff01c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.745+0000 7ff021c73700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 0x7ff01c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45480/0 (socket says 172.21.15.26:45480) 2024-04-21T10:22:07.311 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.745+0000 7ff021c73700 1 -- 172.21.15.26:0/1678444609 learned_addr learned my addr 172.21.15.26:0/1678444609 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.745+0000 7ff021c73700 1 -- 172.21.15.26:0/1678444609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c19ca60 con 0x7ff01c106940 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.745+0000 7ff021c73700 1 --2- 172.21.15.26:0/1678444609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 0x7ff01c19c520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff0180119f0 tx=0x7ff018011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.746+0000 7ff012ffd700 1 -- 172.21.15.26:0/1678444609 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0180039e0 con 0x7ff01c106940 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.746+0000 7ff023ed7700 1 -- 172.21.15.26:0/1678444609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01c19ccf0 con 0x7ff01c106940 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.746+0000 7ff012ffd700 1 -- 172.21.15.26:0/1678444609 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff01801b8f0 con 0x7ff01c106940 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.746+0000 7ff012ffd700 1 -- 172.21.15.26:0/1678444609 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0180237a0 con 0x7ff01c106940 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.746+0000 7ff023ed7700 1 -- 172.21.15.26:0/1678444609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01c19f9e0 con 0x7ff01c106940 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.747+0000 7ff012ffd700 1 -- 172.21.15.26:0/1678444609 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7ff01802c3f0 con 0x7ff01c106940 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.747+0000 7ff012ffd700 1 --2- 172.21.15.26:0/1678444609 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff0080395b0 0x7ff00803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.747+0000 7ff012ffd700 1 -- 172.21.15.26:0/1678444609 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff01804ef80 con 0x7ff01c106940 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.747+0000 7ff021472700 1 --2- 172.21.15.26:0/1678444609 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff0080395b0 0x7ff00803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.747+0000 7ff023ed7700 1 -- 172.21.15.26:0/1678444609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff01c0640a0 con 0x7ff01c106940 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.747+0000 7ff021472700 1 --2- 172.21.15.26:0/1678444609 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff0080395b0 0x7ff00803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff00c006fd0 tx=0x7ff00c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:07.312 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.753+0000 7ff012ffd700 1 -- 172.21.15.26:0/1678444609 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff01801f030 con 0x7ff01c106940 2024-04-21T10:22:07.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:05.989+0000 7ff023ed7700 1 -- 172.21.15.26:0/1678444609 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ff01c103480 con 0x7ff0080395b0 2024-04-21T10:22:07.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:06.235+0000 7ff012ffd700 1 -- 172.21.15.26:0/1678444609 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ff01c103480 con 0x7ff0080395b0 2024-04-21T10:22:07.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:06.238+0000 7ff023ed7700 1 -- 172.21.15.26:0/1678444609 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff0080395b0 msgr2=0x7ff00803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:07.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:06.238+0000 7ff023ed7700 1 --2- 172.21.15.26:0/1678444609 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff0080395b0 0x7ff00803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff00c006fd0 tx=0x7ff00c006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:07.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:06.238+0000 7ff023ed7700 1 -- 172.21.15.26:0/1678444609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 msgr2=0x7ff01c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:07.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:06.238+0000 7ff023ed7700 1 --2- 172.21.15.26:0/1678444609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 0x7ff01c19c520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff0180119f0 tx=0x7ff018011ad0 comp rx=0 tx=0).stop 2024-04-21T10:22:07.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:06.238+0000 7ff023ed7700 1 -- 172.21.15.26:0/1678444609 shutdown_connections 2024-04-21T10:22:07.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:06.238+0000 7ff023ed7700 1 --2- 172.21.15.26:0/1678444609 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff0080395b0 0x7ff00803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:07.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:06.238+0000 7ff023ed7700 1 --2- 172.21.15.26:0/1678444609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff01c106940 0x7ff01c19c520 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:07.313 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:06.238+0000 7ff023ed7700 1 -- 172.21.15.26:0/1678444609 >> 172.21.15.26:0/1678444609 conn(0x7ff01c1022b0 msgr2=0x7ff01c102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:07.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:06.238+0000 7ff023ed7700 1 -- 172.21.15.26:0/1678444609 shutdown_connections 2024-04-21T10:22:07.314 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:06.238+0000 7ff023ed7700 1 -- 172.21.15.26:0/1678444609 wait complete. 2024-04-21T10:22:07.314 INFO:teuthology.orchestra.run.smithi026.stdout:Fetching dashboard port number... 2024-04-21T10:22:07.562 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:07 smithi026 ceph-mon[27965]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:22:07.562 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:07 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:08.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:08 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1563807439' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-21T10:22:08.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 8443 2024-04-21T10:22:08.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.734+0000 7f05507c3700 1 Processor -- start 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.734+0000 7f05507c3700 1 -- start start 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.734+0000 7f05507c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 0x7f054810b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.734+0000 7f05507c3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f054807ce70 con 0x7f05481090c0 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.735+0000 7f054e55f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 0x7f054810b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.735+0000 7f054e55f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 0x7f054810b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45496/0 (socket says 172.21.15.26:45496) 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.735+0000 7f054e55f700 1 -- 172.21.15.26:0/1565696397 learned_addr learned my addr 172.21.15.26:0/1565696397 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.735+0000 7f054e55f700 1 -- 172.21.15.26:0/1565696397 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054810b9f0 con 0x7f05481090c0 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.735+0000 7f054e55f700 1 --2- 172.21.15.26:0/1565696397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 0x7f054810b4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0540009e80 tx=0x7f054000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43995d6b052597d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.735+0000 7f054d55d700 1 -- 172.21.15.26:0/1565696397 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0540017070 con 0x7f05481090c0 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.735+0000 7f054d55d700 1 -- 172.21.15.26:0/1565696397 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f054000b8c0 con 0x7f05481090c0 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.735+0000 7f054d55d700 1 -- 172.21.15.26:0/1565696397 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0540004490 con 0x7f05481090c0 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.736+0000 7f05507c3700 1 -- 172.21.15.26:0/1565696397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 msgr2=0x7f054810b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:08.533 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.736+0000 7f05507c3700 1 --2- 172.21.15.26:0/1565696397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 0x7f054810b4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0540009e80 tx=0x7f054000b330 comp rx=0 tx=0).stop 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.736+0000 7f05507c3700 1 -- 172.21.15.26:0/1565696397 shutdown_connections 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.736+0000 7f05507c3700 1 --2- 172.21.15.26:0/1565696397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 0x7f054810b4b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.736+0000 7f05507c3700 1 -- 172.21.15.26:0/1565696397 >> 172.21.15.26:0/1565696397 conn(0x7f0548102b10 msgr2=0x7f0548104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.736+0000 7f05507c3700 1 -- 172.21.15.26:0/1565696397 shutdown_connections 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.736+0000 7f05507c3700 1 -- 172.21.15.26:0/1565696397 wait complete. 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.737+0000 7f05507c3700 1 Processor -- start 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.737+0000 7f05507c3700 1 -- start start 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.737+0000 7f05507c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 0x7f0548108940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.737+0000 7f05507c3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f054810c9f0 con 0x7f05481090c0 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.737+0000 7f054e55f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 0x7f0548108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.737+0000 7f054e55f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 0x7f0548108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45510/0 (socket says 172.21.15.26:45510) 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.737+0000 7f054e55f700 1 -- 172.21.15.26:0/1563807439 learned_addr learned my addr 172.21.15.26:0/1563807439 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.737+0000 7f054e55f700 1 -- 172.21.15.26:0/1563807439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0548108e80 con 0x7f05481090c0 2024-04-21T10:22:08.534 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.738+0000 7f054e55f700 1 --2- 172.21.15.26:0/1563807439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 0x7f0548108940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f054000b7b0 tx=0x7f054000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.738+0000 7f05477fe700 1 -- 172.21.15.26:0/1563807439 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0540020020 con 0x7f05481090c0 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.738+0000 7f05477fe700 1 -- 172.21.15.26:0/1563807439 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0540003c60 con 0x7f05481090c0 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.738+0000 7f05507c3700 1 -- 172.21.15.26:0/1563807439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0548107070 con 0x7f05481090c0 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.738+0000 7f05477fe700 1 -- 172.21.15.26:0/1563807439 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0540017040 con 0x7f05481090c0 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.738+0000 7f05507c3700 1 -- 172.21.15.26:0/1563807439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05481075b0 con 0x7f05481090c0 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.739+0000 7f05477fe700 1 -- 172.21.15.26:0/1563807439 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f0540023020 con 0x7f05481090c0 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.739+0000 7f05507c3700 1 -- 172.21.15.26:0/1563807439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05480640a0 con 0x7f05481090c0 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.739+0000 7f05477fe700 1 --2- 172.21.15.26:0/1563807439 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0530039660 0x7f053003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.739+0000 7f05477fe700 1 -- 172.21.15.26:0/1563807439 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f054000f050 con 0x7f05481090c0 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.739+0000 7f054dd5e700 1 --2- 172.21.15.26:0/1563807439 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0530039660 0x7f053003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.739+0000 7f054dd5e700 1 --2- 172.21.15.26:0/1563807439 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0530039660 0x7f053003bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f053c006fd0 tx=0x7f053c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.744+0000 7f05477fe700 1 -- 172.21.15.26:0/1563807439 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f054002a020 con 0x7f05481090c0 2024-04-21T10:22:08.535 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.951+0000 7f05507c3700 1 -- 172.21.15.26:0/1563807439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f05481078f0 con 0x7f05481090c0 2024-04-21T10:22:08.536 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.951+0000 7f05477fe700 1 -- 172.21.15.26:0/1563807439 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f05481078f0 con 0x7f05481090c0 2024-04-21T10:22:08.536 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.953+0000 7f05507c3700 1 -- 172.21.15.26:0/1563807439 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0530039660 msgr2=0x7f053003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:08.536 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.953+0000 7f05507c3700 1 --2- 172.21.15.26:0/1563807439 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0530039660 0x7f053003bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f053c006fd0 tx=0x7f053c006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:08.536 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.953+0000 7f05507c3700 1 -- 172.21.15.26:0/1563807439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 msgr2=0x7f0548108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:08.536 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.953+0000 7f05507c3700 1 --2- 172.21.15.26:0/1563807439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 0x7f0548108940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f054000b7b0 tx=0x7f054000bd50 comp rx=0 tx=0).stop 2024-04-21T10:22:08.536 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.953+0000 7f05507c3700 1 -- 172.21.15.26:0/1563807439 shutdown_connections 2024-04-21T10:22:08.536 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.953+0000 7f05507c3700 1 --2- 172.21.15.26:0/1563807439 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0530039660 0x7f053003bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:08.536 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.953+0000 7f05507c3700 1 --2- 172.21.15.26:0/1563807439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05481090c0 0x7f0548108940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:08.536 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.953+0000 7f05507c3700 1 -- 172.21.15.26:0/1563807439 >> 172.21.15.26:0/1563807439 conn(0x7f0548102b10 msgr2=0x7f05481035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:08.536 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.954+0000 7f05507c3700 1 -- 172.21.15.26:0/1563807439 shutdown_connections 2024-04-21T10:22:08.536 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:07.954+0000 7f05507c3700 1 -- 172.21.15.26:0/1563807439 wait complete. 2024-04-21T10:22:08.540 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T10:22:08.540 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2024-04-21T10:22:08.547 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T10:22:08.547 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2024-04-21T10:22:08.547 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2024-04-21T10:22:08.547 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-21T10:22:08.549 INFO:teuthology.orchestra.run.smithi026.stdout:Ceph Dashboard is now available at: 2024-04-21T10:22:08.549 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:22:08.549 INFO:teuthology.orchestra.run.smithi026.stdout: URL: https://smithi026:8443/ 2024-04-21T10:22:08.549 INFO:teuthology.orchestra.run.smithi026.stdout: User: admin 2024-04-21T10:22:08.549 INFO:teuthology.orchestra.run.smithi026.stdout: Password: hfop0c94j1 2024-04-21T10:22:08.549 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:22:08.550 INFO:teuthology.orchestra.run.smithi026.stdout:Saving cluster configuration to /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config directory 2024-04-21T10:22:08.550 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling autotune for osd_memory_target 2024-04-21T10:22:09.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:09 smithi026 ceph-mon[27965]: mgrmap e13: x(active, since 6s) 2024-04-21T10:22:09.621 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.983+0000 7ff057831700 1 Processor -- start 2024-04-21T10:22:09.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.983+0000 7ff057831700 1 -- start start 2024-04-21T10:22:09.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.983+0000 7ff057831700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 0x7ff050108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:09.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.983+0000 7ff057831700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff050109060 con 0x7ff050108670 2024-04-21T10:22:09.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.983+0000 7ff0555cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 0x7ff050108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:09.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.983+0000 7ff0555cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 0x7ff050108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45524/0 (socket says 172.21.15.26:45524) 2024-04-21T10:22:09.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.983+0000 7ff0555cd700 1 -- 172.21.15.26:0/852563082 learned_addr learned my addr 172.21.15.26:0/852563082 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:09.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.984+0000 7ff0555cd700 1 -- 172.21.15.26:0/852563082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff050109820 con 0x7ff050108670 2024-04-21T10:22:09.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.984+0000 7ff0555cd700 1 --2- 172.21.15.26:0/852563082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 0x7ff050108a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0098a0 tx=0x7ff04c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e440581467af6e0b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:09.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.984+0000 7ff047fff700 1 -- 172.21.15.26:0/852563082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff04c004160 con 0x7ff050108670 2024-04-21T10:22:09.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.984+0000 7ff047fff700 1 -- 172.21.15.26:0/852563082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff04c004300 con 0x7ff050108670 2024-04-21T10:22:09.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.984+0000 7ff047fff700 1 -- 172.21.15.26:0/852563082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff04c011a20 con 0x7ff050108670 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.985+0000 7ff057831700 1 -- 172.21.15.26:0/852563082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 msgr2=0x7ff050108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.985+0000 7ff057831700 1 --2- 172.21.15.26:0/852563082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 0x7ff050108a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0098a0 tx=0x7ff04c009bb0 comp rx=0 tx=0).stop 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.985+0000 7ff057831700 1 -- 172.21.15.26:0/852563082 shutdown_connections 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.985+0000 7ff057831700 1 --2- 172.21.15.26:0/852563082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 0x7ff050108a90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.985+0000 7ff057831700 1 -- 172.21.15.26:0/852563082 >> 172.21.15.26:0/852563082 conn(0x7ff050103c30 msgr2=0x7ff050106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.985+0000 7ff057831700 1 -- 172.21.15.26:0/852563082 shutdown_connections 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.985+0000 7ff057831700 1 -- 172.21.15.26:0/852563082 wait complete. 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.985+0000 7ff057831700 1 Processor -- start 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.985+0000 7ff057831700 1 -- start start 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.985+0000 7ff057831700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 0x7ff05007eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.985+0000 7ff057831700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff05010cbd0 con 0x7ff050108670 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.986+0000 7ff0555cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 0x7ff05007eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:09.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.986+0000 7ff0555cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 0x7ff05007eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45536/0 (socket says 172.21.15.26:45536) 2024-04-21T10:22:09.625 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.986+0000 7ff0555cd700 1 -- 172.21.15.26:0/1573179103 learned_addr learned my addr 172.21.15.26:0/1573179103 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:09.625 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.986+0000 7ff0555cd700 1 -- 172.21.15.26:0/1573179103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05007f0b0 con 0x7ff050108670 2024-04-21T10:22:09.625 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.986+0000 7ff0555cd700 1 --2- 172.21.15.26:0/1573179103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 0x7ff05007eb70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0119f0 tx=0x7ff04c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:09.625 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.986+0000 7ff0467fc700 1 -- 172.21.15.26:0/1573179103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff04c0039e0 con 0x7ff050108670 2024-04-21T10:22:09.625 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.986+0000 7ff0467fc700 1 -- 172.21.15.26:0/1573179103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff04c01b8f0 con 0x7ff050108670 2024-04-21T10:22:09.625 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.986+0000 7ff057831700 1 -- 172.21.15.26:0/1573179103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05007d2a0 con 0x7ff050108670 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.986+0000 7ff0467fc700 1 -- 172.21.15.26:0/1573179103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff04c0237a0 con 0x7ff050108670 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.986+0000 7ff057831700 1 -- 172.21.15.26:0/1573179103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05007d7e0 con 0x7ff050108670 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.987+0000 7ff057831700 1 -- 172.21.15.26:0/1573179103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0500640a0 con 0x7ff050108670 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.987+0000 7ff0467fc700 1 -- 172.21.15.26:0/1573179103 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff04c01b440 con 0x7ff050108670 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.988+0000 7ff0467fc700 1 --2- 172.21.15.26:0/1573179103 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff03c0396b0 0x7ff03c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.988+0000 7ff0467fc700 1 -- 172.21.15.26:0/1573179103 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff04c04e060 con 0x7ff050108670 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.988+0000 7ff054dcc700 1 --2- 172.21.15.26:0/1573179103 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff03c0396b0 0x7ff03c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.988+0000 7ff054dcc700 1 --2- 172.21.15.26:0/1573179103 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff03c0396b0 0x7ff03c03bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff040006fd0 tx=0x7ff040006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:08.993+0000 7ff0467fc700 1 -- 172.21.15.26:0/1573179103 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff04c01f030 con 0x7ff050108670 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:09.199+0000 7ff057831700 1 -- 172.21.15.26:0/1573179103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff05007da60 con 0x7ff050108670 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:09.200+0000 7ff0467fc700 1 -- 172.21.15.26:0/1573179103 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff04c018b90 con 0x7ff050108670 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:09.202+0000 7ff057831700 1 -- 172.21.15.26:0/1573179103 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff03c0396b0 msgr2=0x7ff03c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:09.202+0000 7ff057831700 1 --2- 172.21.15.26:0/1573179103 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff03c0396b0 0x7ff03c03bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff040006fd0 tx=0x7ff040006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:09.626 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:09.202+0000 7ff057831700 1 -- 172.21.15.26:0/1573179103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 msgr2=0x7ff05007eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:09.627 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:09.202+0000 7ff057831700 1 --2- 172.21.15.26:0/1573179103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 0x7ff05007eb70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0119f0 tx=0x7ff04c011ad0 comp rx=0 tx=0).stop 2024-04-21T10:22:09.627 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:09.202+0000 7ff057831700 1 -- 172.21.15.26:0/1573179103 shutdown_connections 2024-04-21T10:22:09.627 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:09.202+0000 7ff057831700 1 --2- 172.21.15.26:0/1573179103 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff03c0396b0 0x7ff03c03bb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:09.627 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:09.202+0000 7ff057831700 1 --2- 172.21.15.26:0/1573179103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff050108670 0x7ff05007eb70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:09.627 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:09.202+0000 7ff057831700 1 -- 172.21.15.26:0/1573179103 >> 172.21.15.26:0/1573179103 conn(0x7ff050103c30 msgr2=0x7ff050104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:09.627 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:09.202+0000 7ff057831700 1 -- 172.21.15.26:0/1573179103 shutdown_connections 2024-04-21T10:22:09.627 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:09.202+0000 7ff057831700 1 -- 172.21.15.26:0/1573179103 wait complete. 2024-04-21T10:22:11.028 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.116+0000 7fc8d3b83700 1 Processor -- start 2024-04-21T10:22:11.028 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.116+0000 7fc8d3b83700 1 -- start start 2024-04-21T10:22:11.028 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.116+0000 7fc8d3b83700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 0x7fc8cc108a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:11.028 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.116+0000 7fc8d3b83700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc109050 con 0x7fc8cc108660 2024-04-21T10:22:11.028 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.117+0000 7fc8d191f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 0x7fc8cc108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:11.028 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.117+0000 7fc8d191f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 0x7fc8cc108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45544/0 (socket says 172.21.15.26:45544) 2024-04-21T10:22:11.028 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.117+0000 7fc8d191f700 1 -- 172.21.15.26:0/1369242821 learned_addr learned my addr 172.21.15.26:0/1369242821 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:11.028 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.117+0000 7fc8d191f700 1 -- 172.21.15.26:0/1369242821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8cc109810 con 0x7fc8cc108660 2024-04-21T10:22:11.029 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.117+0000 7fc8d191f700 1 --2- 172.21.15.26:0/1369242821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 0x7fc8cc108a80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009e80 tx=0x7fc8bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373d50e658201217 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:11.029 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.117+0000 7fc8d091d700 1 -- 172.21.15.26:0/1369242821 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8bc017070 con 0x7fc8cc108660 2024-04-21T10:22:11.029 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.117+0000 7fc8d091d700 1 -- 172.21.15.26:0/1369242821 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8bc00b8c0 con 0x7fc8cc108660 2024-04-21T10:22:11.029 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.117+0000 7fc8d091d700 1 -- 172.21.15.26:0/1369242821 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8bc004490 con 0x7fc8cc108660 2024-04-21T10:22:11.029 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.118+0000 7fc8d3b83700 1 -- 172.21.15.26:0/1369242821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 msgr2=0x7fc8cc108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:11.029 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.118+0000 7fc8d3b83700 1 --2- 172.21.15.26:0/1369242821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 0x7fc8cc108a80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009e80 tx=0x7fc8bc00b330 comp rx=0 tx=0).stop 2024-04-21T10:22:11.029 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.118+0000 7fc8d3b83700 1 -- 172.21.15.26:0/1369242821 shutdown_connections 2024-04-21T10:22:11.029 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.118+0000 7fc8d3b83700 1 --2- 172.21.15.26:0/1369242821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 0x7fc8cc108a80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:11.029 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.118+0000 7fc8d3b83700 1 -- 172.21.15.26:0/1369242821 >> 172.21.15.26:0/1369242821 conn(0x7fc8cc103c40 msgr2=0x7fc8cc106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:11.029 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.118+0000 7fc8d3b83700 1 -- 172.21.15.26:0/1369242821 shutdown_connections 2024-04-21T10:22:11.029 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.118+0000 7fc8d3b83700 1 -- 172.21.15.26:0/1369242821 wait complete. 2024-04-21T10:22:11.029 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.119+0000 7fc8d3b83700 1 Processor -- start 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.119+0000 7fc8d3b83700 1 -- start start 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.119+0000 7fc8d3b83700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 0x7fc8cc19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.119+0000 7fc8d3b83700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc10cbc0 con 0x7fc8cc108660 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.119+0000 7fc8d191f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 0x7fc8cc19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.119+0000 7fc8d191f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 0x7fc8cc19c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45558/0 (socket says 172.21.15.26:45558) 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.119+0000 7fc8d191f700 1 -- 172.21.15.26:0/3785891166 learned_addr learned my addr 172.21.15.26:0/3785891166 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.119+0000 7fc8d191f700 1 -- 172.21.15.26:0/3785891166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8cc19ca50 con 0x7fc8cc108660 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.119+0000 7fc8d191f700 1 --2- 172.21.15.26:0/3785891166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 0x7fc8cc19c510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00bca0 tx=0x7fc8bc00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.119+0000 7fc8caffd700 1 -- 172.21.15.26:0/3785891166 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8bc020020 con 0x7fc8cc108660 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.120+0000 7fc8caffd700 1 -- 172.21.15.26:0/3785891166 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8bc003b50 con 0x7fc8cc108660 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.120+0000 7fc8d3b83700 1 -- 172.21.15.26:0/3785891166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8cc19cce0 con 0x7fc8cc108660 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.120+0000 7fc8caffd700 1 -- 172.21.15.26:0/3785891166 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8bc017040 con 0x7fc8cc108660 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.120+0000 7fc8d3b83700 1 -- 172.21.15.26:0/3785891166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8cc19f9d0 con 0x7fc8cc108660 2024-04-21T10:22:11.030 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.120+0000 7fc8caffd700 1 -- 172.21.15.26:0/3785891166 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc8bc023020 con 0x7fc8cc108660 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.120+0000 7fc8d3b83700 1 -- 172.21.15.26:0/3785891166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8cc10cd40 con 0x7fc8cc108660 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.121+0000 7fc8caffd700 1 --2- 172.21.15.26:0/3785891166 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc8b40396f0 0x7fc8b403bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.121+0000 7fc8caffd700 1 -- 172.21.15.26:0/3785891166 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc8bc00f050 con 0x7fc8cc108660 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.121+0000 7fc8d111e700 1 --2- 172.21.15.26:0/3785891166 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc8b40396f0 0x7fc8b403bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.121+0000 7fc8d111e700 1 --2- 172.21.15.26:0/3785891166 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc8b40396f0 0x7fc8b403bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0006fd0 tx=0x7fc8c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.126+0000 7fc8caffd700 1 -- 172.21.15.26:0/3785891166 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8bc02a020 con 0x7fc8cc108660 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.450+0000 7fc8d3b83700 1 -- 172.21.15.26:0/3785891166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc8cc19fba0 con 0x7fc8cc108660 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.452+0000 7fc8caffd700 1 -- 172.21.15.26:0/3785891166 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7fc8cc19fba0 con 0x7fc8cc108660 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.454+0000 7fc8d3b83700 1 -- 172.21.15.26:0/3785891166 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc8b40396f0 msgr2=0x7fc8b403bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.454+0000 7fc8d3b83700 1 --2- 172.21.15.26:0/3785891166 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc8b40396f0 0x7fc8b403bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0006fd0 tx=0x7fc8c0006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.454+0000 7fc8d3b83700 1 -- 172.21.15.26:0/3785891166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 msgr2=0x7fc8cc19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.454+0000 7fc8d3b83700 1 --2- 172.21.15.26:0/3785891166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 0x7fc8cc19c510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00bca0 tx=0x7fc8bc00bd80 comp rx=0 tx=0).stop 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.454+0000 7fc8d3b83700 1 -- 172.21.15.26:0/3785891166 shutdown_connections 2024-04-21T10:22:11.031 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.454+0000 7fc8d3b83700 1 --2- 172.21.15.26:0/3785891166 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc8b40396f0 0x7fc8b403bbb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.454+0000 7fc8d3b83700 1 --2- 172.21.15.26:0/3785891166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8cc108660 0x7fc8cc19c510 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.454+0000 7fc8d3b83700 1 -- 172.21.15.26:0/3785891166 >> 172.21.15.26:0/3785891166 conn(0x7fc8cc103c40 msgr2=0x7fc8cc104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.454+0000 7fc8d3b83700 1 -- 172.21.15.26:0/3785891166 shutdown_connections 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2024-04-21T10:22:10.454+0000 7fc8d3b83700 1 -- 172.21.15.26:0/3785891166 wait complete. 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout:Or, if you are only running a single cluster on this host: 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-21T10:22:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:22:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: ceph telemetry on 2024-04-21T10:22:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:22:11.033 INFO:teuthology.orchestra.run.smithi026.stdout:For more information see: 2024-04-21T10:22:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:22:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-21T10:22:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:22:11.033 INFO:teuthology.orchestra.run.smithi026.stdout:Bootstrap complete. 2024-04-21T10:22:11.068 INFO:tasks.cephadm:Fetching config... 2024-04-21T10:22:11.068 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:22:11.068 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-21T10:22:11.084 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-21T10:22:11.084 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:22:11.084 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-21T10:22:11.140 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-21T10:22:11.140 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:22:11.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-21T10:22:11.205 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-21T10:22:11.205 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:22:11.205 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-21T10:22:11.260 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-21T10:22:11.260 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPRhQDwVyQytoBWYQAnXL/X2ytGQgq/Fv6haEcFIxCmRLyCk2kw4tqgM4YQlToXJmnEytBSCLIKhqk63AsN2VqlrZIo5U3UXy+Hj75dJVgf8NfRu+q/y4bpbmBIjbcsdCaBI6/4R9doeodvs2ZiVddqpNGsFLixG8bywiadrgAS8/J+Qke++wOqTPAPNkj5kIHg0XXk5SsC1SotBC2VG5hkp268+CVKFH4Rrci89D6rf4LGrJMHbUUe2h3tvbmry5n25eQXRBZTiOG0Gw4txefr5TWpgixKfBWahEwQFIE1swpSdKjpn01szpSUnYl/6/4e5pmxUmD5D+Tm3tfqiFM3F1kWtR9oXwD0bjvBgvgbuDXEDAhBVZ9xaJyGQT2C+F8g6oYrLUN0+PnL3LvF5m5qs88Wvbnzf2Nvrj75WT1vSeXi6iLQ4b9LGNWQLcto34V7tukuMAqxp9Dgyp+a1ZlwRD14iPAi6tVqls4L0hS0LkZcaWzGsgXfHF3CpBU9KM= ceph-b77245c8-ffc8-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T10:22:11.336 INFO:teuthology.orchestra.run.smithi026.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPRhQDwVyQytoBWYQAnXL/X2ytGQgq/Fv6haEcFIxCmRLyCk2kw4tqgM4YQlToXJmnEytBSCLIKhqk63AsN2VqlrZIo5U3UXy+Hj75dJVgf8NfRu+q/y4bpbmBIjbcsdCaBI6/4R9doeodvs2ZiVddqpNGsFLixG8bywiadrgAS8/J+Qke++wOqTPAPNkj5kIHg0XXk5SsC1SotBC2VG5hkp268+CVKFH4Rrci89D6rf4LGrJMHbUUe2h3tvbmry5n25eQXRBZTiOG0Gw4txefr5TWpgixKfBWahEwQFIE1swpSdKjpn01szpSUnYl/6/4e5pmxUmD5D+Tm3tfqiFM3F1kWtR9oXwD0bjvBgvgbuDXEDAhBVZ9xaJyGQT2C+F8g6oYrLUN0+PnL3LvF5m5qs88Wvbnzf2Nvrj75WT1vSeXi6iLQ4b9LGNWQLcto34V7tukuMAqxp9Dgyp+a1ZlwRD14iPAi6tVqls4L0hS0LkZcaWzGsgXfHF3CpBU9KM= ceph-b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:22:11.347 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPRhQDwVyQytoBWYQAnXL/X2ytGQgq/Fv6haEcFIxCmRLyCk2kw4tqgM4YQlToXJmnEytBSCLIKhqk63AsN2VqlrZIo5U3UXy+Hj75dJVgf8NfRu+q/y4bpbmBIjbcsdCaBI6/4R9doeodvs2ZiVddqpNGsFLixG8bywiadrgAS8/J+Qke++wOqTPAPNkj5kIHg0XXk5SsC1SotBC2VG5hkp268+CVKFH4Rrci89D6rf4LGrJMHbUUe2h3tvbmry5n25eQXRBZTiOG0Gw4txefr5TWpgixKfBWahEwQFIE1swpSdKjpn01szpSUnYl/6/4e5pmxUmD5D+Tm3tfqiFM3F1kWtR9oXwD0bjvBgvgbuDXEDAhBVZ9xaJyGQT2C+F8g6oYrLUN0+PnL3LvF5m5qs88Wvbnzf2Nvrj75WT1vSeXi6iLQ4b9LGNWQLcto34V7tukuMAqxp9Dgyp+a1ZlwRD14iPAi6tVqls4L0hS0LkZcaWzGsgXfHF3CpBU9KM= ceph-b77245c8-ffc8-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T10:22:11.386 INFO:teuthology.orchestra.run.smithi169.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPRhQDwVyQytoBWYQAnXL/X2ytGQgq/Fv6haEcFIxCmRLyCk2kw4tqgM4YQlToXJmnEytBSCLIKhqk63AsN2VqlrZIo5U3UXy+Hj75dJVgf8NfRu+q/y4bpbmBIjbcsdCaBI6/4R9doeodvs2ZiVddqpNGsFLixG8bywiadrgAS8/J+Qke++wOqTPAPNkj5kIHg0XXk5SsC1SotBC2VG5hkp268+CVKFH4Rrci89D6rf4LGrJMHbUUe2h3tvbmry5n25eQXRBZTiOG0Gw4txefr5TWpgixKfBWahEwQFIE1swpSdKjpn01szpSUnYl/6/4e5pmxUmD5D+Tm3tfqiFM3F1kWtR9oXwD0bjvBgvgbuDXEDAhBVZ9xaJyGQT2C+F8g6oYrLUN0+PnL3LvF5m5qs88Wvbnzf2Nvrj75WT1vSeXi6iLQ4b9LGNWQLcto34V7tukuMAqxp9Dgyp+a1ZlwRD14iPAi6tVqls4L0hS0LkZcaWzGsgXfHF3CpBU9KM= ceph-b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:22:11.398 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPRhQDwVyQytoBWYQAnXL/X2ytGQgq/Fv6haEcFIxCmRLyCk2kw4tqgM4YQlToXJmnEytBSCLIKhqk63AsN2VqlrZIo5U3UXy+Hj75dJVgf8NfRu+q/y4bpbmBIjbcsdCaBI6/4R9doeodvs2ZiVddqpNGsFLixG8bywiadrgAS8/J+Qke++wOqTPAPNkj5kIHg0XXk5SsC1SotBC2VG5hkp268+CVKFH4Rrci89D6rf4LGrJMHbUUe2h3tvbmry5n25eQXRBZTiOG0Gw4txefr5TWpgixKfBWahEwQFIE1swpSdKjpn01szpSUnYl/6/4e5pmxUmD5D+Tm3tfqiFM3F1kWtR9oXwD0bjvBgvgbuDXEDAhBVZ9xaJyGQT2C+F8g6oYrLUN0+PnL3LvF5m5qs88Wvbnzf2Nvrj75WT1vSeXi6iLQ4b9LGNWQLcto34V7tukuMAqxp9Dgyp+a1ZlwRD14iPAi6tVqls4L0hS0LkZcaWzGsgXfHF3CpBU9KM= ceph-b77245c8-ffc8-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T10:22:11.437 INFO:teuthology.orchestra.run.smithi190.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPRhQDwVyQytoBWYQAnXL/X2ytGQgq/Fv6haEcFIxCmRLyCk2kw4tqgM4YQlToXJmnEytBSCLIKhqk63AsN2VqlrZIo5U3UXy+Hj75dJVgf8NfRu+q/y4bpbmBIjbcsdCaBI6/4R9doeodvs2ZiVddqpNGsFLixG8bywiadrgAS8/J+Qke++wOqTPAPNkj5kIHg0XXk5SsC1SotBC2VG5hkp268+CVKFH4Rrci89D6rf4LGrJMHbUUe2h3tvbmry5n25eQXRBZTiOG0Gw4txefr5TWpgixKfBWahEwQFIE1swpSdKjpn01szpSUnYl/6/4e5pmxUmD5D+Tm3tfqiFM3F1kWtR9oXwD0bjvBgvgbuDXEDAhBVZ9xaJyGQT2C+F8g6oYrLUN0+PnL3LvF5m5qs88Wvbnzf2Nvrj75WT1vSeXi6iLQ4b9LGNWQLcto34V7tukuMAqxp9Dgyp+a1ZlwRD14iPAi6tVqls4L0hS0LkZcaWzGsgXfHF3CpBU9KM= ceph-b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:22:11.449 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-21T10:22:11.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:11 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3785891166' entity='client.admin' 2024-04-21T10:22:11.689 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:22:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.427+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1658808590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e01024e0 msgr2=0x7f43e0102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.427+0000 7f43e7d6c700 1 --2- 172.21.15.26:0/1658808590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e01024e0 0x7f43e0102900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009910 tx=0x7f43dc009c20 comp rx=0 tx=0).stop 2024-04-21T10:22:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.428+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1658808590 shutdown_connections 2024-04-21T10:22:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.428+0000 7f43e7d6c700 1 --2- 172.21.15.26:0/1658808590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e01024e0 0x7f43e0102900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.428+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1658808590 >> 172.21.15.26:0/1658808590 conn(0x7f43e00fda40 msgr2=0x7f43e00ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.428+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1658808590 shutdown_connections 2024-04-21T10:22:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.428+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1658808590 wait complete. 2024-04-21T10:22:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.428+0000 7f43e7d6c700 1 Processor -- start 2024-04-21T10:22:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.428+0000 7f43e7d6c700 1 -- start start 2024-04-21T10:22:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.428+0000 7f43e7d6c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e01024e0 0x7f43e00766b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.428+0000 7f43e7d6c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f43e0108420 con 0x7f43e01024e0 2024-04-21T10:22:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.429+0000 7f43e5b08700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e01024e0 0x7f43e00766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.429+0000 7f43e5b08700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e01024e0 0x7f43e00766b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45572/0 (socket says 172.21.15.26:45572) 2024-04-21T10:22:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.429+0000 7f43e5b08700 1 -- 172.21.15.26:0/1230952316 learned_addr learned my addr 172.21.15.26:0/1230952316 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.429+0000 7f43e5b08700 1 -- 172.21.15.26:0/1230952316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e0076bf0 con 0x7f43e01024e0 2024-04-21T10:22:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.429+0000 7f43e5b08700 1 --2- 172.21.15.26:0/1230952316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e01024e0 0x7f43e00766b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f43dc005950 tx=0x7f43dc004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.429+0000 7f43d6ffd700 1 -- 172.21.15.26:0/1230952316 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43dc026070 con 0x7f43e01024e0 2024-04-21T10:22:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.429+0000 7f43d6ffd700 1 -- 172.21.15.26:0/1230952316 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43dc00bc00 con 0x7f43e01024e0 2024-04-21T10:22:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.429+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1230952316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e0074e30 con 0x7f43e01024e0 2024-04-21T10:22:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.429+0000 7f43d6ffd700 1 -- 172.21.15.26:0/1230952316 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43dc003a10 con 0x7f43e01024e0 2024-04-21T10:22:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.429+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1230952316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e0075310 con 0x7f43e01024e0 2024-04-21T10:22:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.430+0000 7f43d6ffd700 1 -- 172.21.15.26:0/1230952316 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f43dc029070 con 0x7f43e01024e0 2024-04-21T10:22:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.430+0000 7f43d6ffd700 1 --2- 172.21.15.26:0/1230952316 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f43cc039650 0x7f43cc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.430+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1230952316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e0108ba0 con 0x7f43e01024e0 2024-04-21T10:22:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.430+0000 7f43d6ffd700 1 -- 172.21.15.26:0/1230952316 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f43dc01e070 con 0x7f43e01024e0 2024-04-21T10:22:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.430+0000 7f43e5307700 1 --2- 172.21.15.26:0/1230952316 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f43cc039650 0x7f43cc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.431+0000 7f43e5307700 1 --2- 172.21.15.26:0/1230952316 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f43cc039650 0x7f43cc03bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f43d0006fd0 tx=0x7f43d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.436+0000 7f43d6ffd700 1 -- 172.21.15.26:0/1230952316 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43dc030070 con 0x7f43e01024e0 2024-04-21T10:22:13.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.643+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1230952316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f43e0190e10 con 0x7f43e01024e0 2024-04-21T10:22:13.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.645+0000 7f43d6ffd700 1 -- 172.21.15.26:0/1230952316 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f43dc013070 con 0x7f43e01024e0 2024-04-21T10:22:13.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.647+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1230952316 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f43cc039650 msgr2=0x7f43cc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:13.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.647+0000 7f43e7d6c700 1 --2- 172.21.15.26:0/1230952316 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f43cc039650 0x7f43cc03bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f43d0006fd0 tx=0x7f43d0006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:13.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.647+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1230952316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e01024e0 msgr2=0x7f43e00766b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:13.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.647+0000 7f43e7d6c700 1 --2- 172.21.15.26:0/1230952316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e01024e0 0x7f43e00766b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f43dc005950 tx=0x7f43dc004fa0 comp rx=0 tx=0).stop 2024-04-21T10:22:13.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.647+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1230952316 shutdown_connections 2024-04-21T10:22:13.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.647+0000 7f43e7d6c700 1 --2- 172.21.15.26:0/1230952316 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f43cc039650 0x7f43cc03bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:13.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.647+0000 7f43e7d6c700 1 --2- 172.21.15.26:0/1230952316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e01024e0 0x7f43e00766b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:13.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.647+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1230952316 >> 172.21.15.26:0/1230952316 conn(0x7f43e00fda40 msgr2=0x7f43e00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:13.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.647+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1230952316 shutdown_connections 2024-04-21T10:22:13.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:13.648+0000 7f43e7d6c700 1 -- 172.21.15.26:0/1230952316 wait complete. 2024-04-21T10:22:14.292 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-21T10:22:14.293 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-21T10:22:14.533 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:22:14.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:14 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1230952316' entity='client.admin' 2024-04-21T10:22:14.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:14.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:14.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:22:14.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:14.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:22:14.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:15.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.988+0000 7fde8381d700 1 -- 172.21.15.26:0/949719631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7c1007a0 msgr2=0x7fde7c100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:15.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.988+0000 7fde8381d700 1 --2- 172.21.15.26:0/949719631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7c1007a0 0x7fde7c100bc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fde7400b0a0 tx=0x7fde7400b3b0 comp rx=0 tx=0).stop 2024-04-21T10:22:15.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.988+0000 7fde8381d700 1 -- 172.21.15.26:0/949719631 shutdown_connections 2024-04-21T10:22:15.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.988+0000 7fde8381d700 1 --2- 172.21.15.26:0/949719631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7c1007a0 0x7fde7c100bc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:15.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.988+0000 7fde8381d700 1 -- 172.21.15.26:0/949719631 >> 172.21.15.26:0/949719631 conn(0x7fde7c0fc0c0 msgr2=0x7fde7c0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:15.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.988+0000 7fde8381d700 1 -- 172.21.15.26:0/949719631 shutdown_connections 2024-04-21T10:22:15.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.988+0000 7fde8381d700 1 -- 172.21.15.26:0/949719631 wait complete. 2024-04-21T10:22:15.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.989+0000 7fde8381d700 1 Processor -- start 2024-04-21T10:22:15.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.989+0000 7fde8381d700 1 -- start start 2024-04-21T10:22:15.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.989+0000 7fde8381d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7c1007a0 0x7fde7c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:15.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.989+0000 7fde8381d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c108420 con 0x7fde7c1007a0 2024-04-21T10:22:15.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.989+0000 7fde815b9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7c1007a0 0x7fde7c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:15.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.989+0000 7fde815b9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7c1007a0 0x7fde7c197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38444/0 (socket says 172.21.15.26:38444) 2024-04-21T10:22:15.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.989+0000 7fde815b9700 1 -- 172.21.15.26:0/2621325583 learned_addr learned my addr 172.21.15.26:0/2621325583 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:15.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.989+0000 7fde815b9700 1 -- 172.21.15.26:0/2621325583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c197fa0 con 0x7fde7c1007a0 2024-04-21T10:22:15.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.989+0000 7fde815b9700 1 --2- 172.21.15.26:0/2621325583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7c1007a0 0x7fde7c197a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fde7400d010 tx=0x7fde74003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:15.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.989+0000 7fde7a7fc700 1 -- 172.21.15.26:0/2621325583 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde74021020 con 0x7fde7c1007a0 2024-04-21T10:22:15.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.990+0000 7fde7a7fc700 1 -- 172.21.15.26:0/2621325583 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7400bdd0 con 0x7fde7c1007a0 2024-04-21T10:22:15.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.990+0000 7fde8381d700 1 -- 172.21.15.26:0/2621325583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c198230 con 0x7fde7c1007a0 2024-04-21T10:22:15.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.990+0000 7fde7a7fc700 1 -- 172.21.15.26:0/2621325583 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde74017040 con 0x7fde7c1007a0 2024-04-21T10:22:15.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.990+0000 7fde8381d700 1 -- 172.21.15.26:0/2621325583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c19af20 con 0x7fde7c1007a0 2024-04-21T10:22:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.990+0000 7fde7a7fc700 1 -- 172.21.15.26:0/2621325583 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fde74024020 con 0x7fde7c1007a0 2024-04-21T10:22:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.990+0000 7fde8381d700 1 -- 172.21.15.26:0/2621325583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7c1986f0 con 0x7fde7c1007a0 2024-04-21T10:22:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.991+0000 7fde7a7fc700 1 --2- 172.21.15.26:0/2621325583 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fde640396b0 0x7fde6403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.991+0000 7fde7a7fc700 1 -- 172.21.15.26:0/2621325583 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fde7400f050 con 0x7fde7c1007a0 2024-04-21T10:22:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.991+0000 7fde80db8700 1 --2- 172.21.15.26:0/2621325583 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fde640396b0 0x7fde6403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.991+0000 7fde80db8700 1 --2- 172.21.15.26:0/2621325583 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fde640396b0 0x7fde6403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fde70006fd0 tx=0x7fde70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:15.996+0000 7fde7a7fc700 1 -- 172.21.15.26:0/2621325583 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde7402b020 con 0x7fde7c1007a0 2024-04-21T10:22:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:16.207+0000 7fde8381d700 1 -- 172.21.15.26:0/2621325583 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fde7c02e090 con 0x7fde640396b0 2024-04-21T10:22:16.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:16.211+0000 7fde7a7fc700 1 -- 172.21.15.26:0/2621325583 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fde7c02e090 con 0x7fde640396b0 2024-04-21T10:22:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:16.213+0000 7fde8381d700 1 -- 172.21.15.26:0/2621325583 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fde640396b0 msgr2=0x7fde6403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:16.213+0000 7fde8381d700 1 --2- 172.21.15.26:0/2621325583 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fde640396b0 0x7fde6403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fde70006fd0 tx=0x7fde70006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:16.213+0000 7fde8381d700 1 -- 172.21.15.26:0/2621325583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7c1007a0 msgr2=0x7fde7c197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:16.213+0000 7fde8381d700 1 --2- 172.21.15.26:0/2621325583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7c1007a0 0x7fde7c197a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fde7400d010 tx=0x7fde74003ce0 comp rx=0 tx=0).stop 2024-04-21T10:22:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:16.213+0000 7fde8381d700 1 -- 172.21.15.26:0/2621325583 shutdown_connections 2024-04-21T10:22:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:16.213+0000 7fde8381d700 1 --2- 172.21.15.26:0/2621325583 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fde640396b0 0x7fde6403bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:16.213+0000 7fde8381d700 1 --2- 172.21.15.26:0/2621325583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7c1007a0 0x7fde7c197a60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:16.213+0000 7fde8381d700 1 -- 172.21.15.26:0/2621325583 >> 172.21.15.26:0/2621325583 conn(0x7fde7c0fc0c0 msgr2=0x7fde7c0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:16.213+0000 7fde8381d700 1 -- 172.21.15.26:0/2621325583 shutdown_connections 2024-04-21T10:22:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:16.213+0000 7fde8381d700 1 -- 172.21.15.26:0/2621325583 wait complete. 2024-04-21T10:22:16.941 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi169 2024-04-21T10:22:16.941 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-21T10:22:16.941 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.conf 2024-04-21T10:22:16.959 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-21T10:22:16.960 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:22:17.017 INFO:tasks.cephadm:Adding host smithi169 to orchestrator... 2024-04-21T10:22:17.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch host add smithi169 2024-04-21T10:22:17.260 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:22:17.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:17 smithi026 ceph-mon[27965]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:22:17.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:17 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:17.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:17 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:22:17.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:17 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:17.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:17 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:22:17.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:17 smithi026 ceph-mon[27965]: Updating smithi026:/etc/ceph/ceph.conf 2024-04-21T10:22:17.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:17 smithi026 ceph-mon[27965]: Updating smithi026:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:22:17.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:17 smithi026 ceph-mon[27965]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:22:17.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:17 smithi026 ceph-mon[27965]: Updating smithi026:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:22:17.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:17 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:17.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:17 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:17.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:17 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.657+0000 7fbe6b578700 1 -- 172.21.15.26:0/1059245768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe641084e0 msgr2=0x7fbe641088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.657+0000 7fbe6b578700 1 --2- 172.21.15.26:0/1059245768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe641084e0 0x7fbe641088c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000b0a0 tx=0x7fbe6000b3b0 comp rx=0 tx=0).stop 2024-04-21T10:22:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.657+0000 7fbe6b578700 1 -- 172.21.15.26:0/1059245768 shutdown_connections 2024-04-21T10:22:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.657+0000 7fbe6b578700 1 --2- 172.21.15.26:0/1059245768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe641084e0 0x7fbe641088c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.657+0000 7fbe6b578700 1 -- 172.21.15.26:0/1059245768 >> 172.21.15.26:0/1059245768 conn(0x7fbe64076ea0 msgr2=0x7fbe640772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.657+0000 7fbe6b578700 1 -- 172.21.15.26:0/1059245768 shutdown_connections 2024-04-21T10:22:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.657+0000 7fbe6b578700 1 -- 172.21.15.26:0/1059245768 wait complete. 2024-04-21T10:22:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.658+0000 7fbe6b578700 1 Processor -- start 2024-04-21T10:22:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.658+0000 7fbe6b578700 1 -- start start 2024-04-21T10:22:18.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.658+0000 7fbe6b578700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe641084e0 0x7fbe6419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:18.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.658+0000 7fbe6b578700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6410c920 con 0x7fbe641084e0 2024-04-21T10:22:18.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.658+0000 7fbe69314700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe641084e0 0x7fbe6419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:18.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.658+0000 7fbe69314700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe641084e0 0x7fbe6419a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38452/0 (socket says 172.21.15.26:38452) 2024-04-21T10:22:18.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.658+0000 7fbe69314700 1 -- 172.21.15.26:0/3273040308 learned_addr learned my addr 172.21.15.26:0/3273040308 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:18.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.658+0000 7fbe69314700 1 -- 172.21.15.26:0/3273040308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6419a560 con 0x7fbe641084e0 2024-04-21T10:22:18.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.659+0000 7fbe69314700 1 --2- 172.21.15.26:0/3273040308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe641084e0 0x7fbe6419a020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000d010 tx=0x7fbe60003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:18.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.659+0000 7fbe5a7fc700 1 -- 172.21.15.26:0/3273040308 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe60021020 con 0x7fbe641084e0 2024-04-21T10:22:18.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.659+0000 7fbe6b578700 1 -- 172.21.15.26:0/3273040308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6419a7f0 con 0x7fbe641084e0 2024-04-21T10:22:18.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.659+0000 7fbe5a7fc700 1 -- 172.21.15.26:0/3273040308 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe600044b0 con 0x7fbe641084e0 2024-04-21T10:22:18.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.659+0000 7fbe5a7fc700 1 -- 172.21.15.26:0/3273040308 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe60017040 con 0x7fbe641084e0 2024-04-21T10:22:18.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.659+0000 7fbe6b578700 1 -- 172.21.15.26:0/3273040308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6419e8f0 con 0x7fbe641084e0 2024-04-21T10:22:18.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.660+0000 7fbe5a7fc700 1 -- 172.21.15.26:0/3273040308 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbe60024020 con 0x7fbe641084e0 2024-04-21T10:22:18.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.660+0000 7fbe6b578700 1 -- 172.21.15.26:0/3273040308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe64103b90 con 0x7fbe641084e0 2024-04-21T10:22:18.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.660+0000 7fbe5a7fc700 1 --2- 172.21.15.26:0/3273040308 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbe500396a0 0x7fbe5003bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:18.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.660+0000 7fbe5a7fc700 1 -- 172.21.15.26:0/3273040308 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbe6000f050 con 0x7fbe641084e0 2024-04-21T10:22:18.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.660+0000 7fbe68b13700 1 --2- 172.21.15.26:0/3273040308 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbe500396a0 0x7fbe5003bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:18.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.660+0000 7fbe68b13700 1 --2- 172.21.15.26:0/3273040308 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbe500396a0 0x7fbe5003bb60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbe54006fd0 tx=0x7fbe54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:18.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.665+0000 7fbe5a7fc700 1 -- 172.21.15.26:0/3273040308 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe6002b020 con 0x7fbe641084e0 2024-04-21T10:22:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:18.876+0000 7fbe6b578700 1 -- 172.21.15.26:0/3273040308 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}) -- 0x7fbe6419b380 con 0x7fbe500396a0 2024-04-21T10:22:19.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:18 smithi026 ceph-mon[27965]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:22:20.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:19 smithi026 ceph-mon[27965]: Deploying cephadm binary to smithi169 2024-04-21T10:22:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:23.070+0000 7fbe5a7fc700 1 -- 172.21.15.26:0/3273040308 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fbe6419b380 con 0x7fbe500396a0 2024-04-21T10:22:23.071 INFO:teuthology.orchestra.run.smithi026.stdout:Added host 'smithi169' with addr '172.21.15.169' 2024-04-21T10:22:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:23.073+0000 7fbe6b578700 1 -- 172.21.15.26:0/3273040308 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbe500396a0 msgr2=0x7fbe5003bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:23.073+0000 7fbe6b578700 1 --2- 172.21.15.26:0/3273040308 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbe500396a0 0x7fbe5003bb60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbe54006fd0 tx=0x7fbe54006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:23.073+0000 7fbe6b578700 1 -- 172.21.15.26:0/3273040308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe641084e0 msgr2=0x7fbe6419a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:23.073+0000 7fbe6b578700 1 --2- 172.21.15.26:0/3273040308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe641084e0 0x7fbe6419a020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000d010 tx=0x7fbe60003ce0 comp rx=0 tx=0).stop 2024-04-21T10:22:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:23.073+0000 7fbe6b578700 1 -- 172.21.15.26:0/3273040308 shutdown_connections 2024-04-21T10:22:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:23.073+0000 7fbe6b578700 1 --2- 172.21.15.26:0/3273040308 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbe500396a0 0x7fbe5003bb60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:23.073+0000 7fbe6b578700 1 --2- 172.21.15.26:0/3273040308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe641084e0 0x7fbe6419a020 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:23.073+0000 7fbe6b578700 1 -- 172.21.15.26:0/3273040308 >> 172.21.15.26:0/3273040308 conn(0x7fbe64076ea0 msgr2=0x7fbe64104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:23.073+0000 7fbe6b578700 1 -- 172.21.15.26:0/3273040308 shutdown_connections 2024-04-21T10:22:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:23.073+0000 7fbe6b578700 1 -- 172.21.15.26:0/3273040308 wait complete. 2024-04-21T10:22:23.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:23 smithi026 ceph-mon[27965]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:23.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:23.389 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:22:23.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-21T10:22:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:22:24.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:24 smithi026 ceph-mon[27965]: Added host smithi169 2024-04-21T10:22:24.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:24 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:24.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:24 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.513+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/4195690680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa8103650 msgr2=0x7f5aa8103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.513+0000 7f5aaf5e1700 1 --2- 172.21.15.26:0/4195690680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa8103650 0x7f5aa8103a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009910 tx=0x7f5aa4009c20 comp rx=0 tx=0).stop 2024-04-21T10:22:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.513+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/4195690680 shutdown_connections 2024-04-21T10:22:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.513+0000 7f5aaf5e1700 1 --2- 172.21.15.26:0/4195690680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa8103650 0x7f5aa8103a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.513+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/4195690680 >> 172.21.15.26:0/4195690680 conn(0x7f5aa80fcea0 msgr2=0x7f5aa80ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.513+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/4195690680 shutdown_connections 2024-04-21T10:22:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.513+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/4195690680 wait complete. 2024-04-21T10:22:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.514+0000 7f5aaf5e1700 1 Processor -- start 2024-04-21T10:22:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.514+0000 7f5aaf5e1700 1 -- start start 2024-04-21T10:22:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.514+0000 7f5aaf5e1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa8103650 0x7f5aa8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.514+0000 7f5aaf5e1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa810c9d0 con 0x7f5aa8103650 2024-04-21T10:22:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.514+0000 7f5aad37d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa8103650 0x7f5aa8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.514+0000 7f5aad37d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa8103650 0x7f5aa8197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54246/0 (socket says 172.21.15.26:54246) 2024-04-21T10:22:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.514+0000 7f5aad37d700 1 -- 172.21.15.26:0/53331020 learned_addr learned my addr 172.21.15.26:0/53331020 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.514+0000 7f5aad37d700 1 -- 172.21.15.26:0/53331020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa8198430 con 0x7f5aa8103650 2024-04-21T10:22:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.515+0000 7f5aad37d700 1 --2- 172.21.15.26:0/53331020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa8103650 0x7f5aa8197ef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40056d0 tx=0x7f5aa40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.515+0000 7f5a9e7fc700 1 -- 172.21.15.26:0/53331020 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aa4026070 con 0x7f5aa8103650 2024-04-21T10:22:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.515+0000 7f5a9e7fc700 1 -- 172.21.15.26:0/53331020 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa4004370 con 0x7f5aa8103650 2024-04-21T10:22:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.515+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/53331020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa81986c0 con 0x7f5aa8103650 2024-04-21T10:22:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.515+0000 7f5a9e7fc700 1 -- 172.21.15.26:0/53331020 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aa40039b0 con 0x7f5aa8103650 2024-04-21T10:22:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.515+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/53331020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa8198b20 con 0x7f5aa8103650 2024-04-21T10:22:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.516+0000 7f5a9e7fc700 1 -- 172.21.15.26:0/53331020 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5aa4029070 con 0x7f5aa8103650 2024-04-21T10:22:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.516+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/53331020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa806b190 con 0x7f5aa8103650 2024-04-21T10:22:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.516+0000 7f5a9e7fc700 1 --2- 172.21.15.26:0/53331020 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f5a940396b0 0x7f5a9403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.516+0000 7f5a9e7fc700 1 -- 172.21.15.26:0/53331020 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5aa401e070 con 0x7f5aa8103650 2024-04-21T10:22:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.516+0000 7f5aacb7c700 1 --2- 172.21.15.26:0/53331020 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f5a940396b0 0x7f5a9403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.517+0000 7f5aacb7c700 1 --2- 172.21.15.26:0/53331020 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f5a940396b0 0x7f5a9403bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5a98006fd0 tx=0x7f5a98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.521+0000 7f5a9e7fc700 1 -- 172.21.15.26:0/53331020 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5aa4013070 con 0x7f5aa8103650 2024-04-21T10:22:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.738+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/53331020 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5aa81992c0 con 0x7f5a940396b0 2024-04-21T10:22:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.739+0000 7f5a9e7fc700 1 -- 172.21.15.26:0/53331020 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f5aa81992c0 con 0x7f5a940396b0 2024-04-21T10:22:25.739 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:22:25.739 INFO:teuthology.orchestra.run.smithi026.stdout:[{"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.169", "hostname": "smithi169", "labels": [], "status": ""}] 2024-04-21T10:22:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.741+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/53331020 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f5a940396b0 msgr2=0x7f5a9403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.741+0000 7f5aaf5e1700 1 --2- 172.21.15.26:0/53331020 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f5a940396b0 0x7f5a9403bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5a98006fd0 tx=0x7f5a98006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.741+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/53331020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa8103650 msgr2=0x7f5aa8197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.741+0000 7f5aaf5e1700 1 --2- 172.21.15.26:0/53331020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa8103650 0x7f5aa8197ef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40056d0 tx=0x7f5aa40057b0 comp rx=0 tx=0).stop 2024-04-21T10:22:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.741+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/53331020 shutdown_connections 2024-04-21T10:22:25.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.741+0000 7f5aaf5e1700 1 --2- 172.21.15.26:0/53331020 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f5a940396b0 0x7f5a9403bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.741+0000 7f5aaf5e1700 1 --2- 172.21.15.26:0/53331020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa8103650 0x7f5aa8197ef0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.741+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/53331020 >> 172.21.15.26:0/53331020 conn(0x7f5aa80fcea0 msgr2=0x7f5aa81043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.741+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/53331020 shutdown_connections 2024-04-21T10:22:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:25.741+0000 7f5aaf5e1700 1 -- 172.21.15.26:0/53331020 wait complete. 2024-04-21T10:22:25.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:25 smithi026 ceph-mon[27965]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:26.360 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi190 2024-04-21T10:22:26.361 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-21T10:22:26.361 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.conf 2024-04-21T10:22:26.379 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-21T10:22:26.379 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:22:26.436 INFO:tasks.cephadm:Adding host smithi190 to orchestrator... 2024-04-21T10:22:26.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch host add smithi190 2024-04-21T10:22:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:22:26.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:26 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:26.909 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:26 smithi026 ceph-mon[27965]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:22:27.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:27 smithi026 ceph-mon[27965]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.105+0000 7f27eab4f700 1 -- 172.21.15.26:0/605620616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e41015f0 msgr2=0x7f27e41019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.105+0000 7f27eab4f700 1 --2- 172.21.15.26:0/605620616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e41015f0 0x7f27e41019d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009910 tx=0x7f27d4009c20 comp rx=0 tx=0).stop 2024-04-21T10:22:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.105+0000 7f27eab4f700 1 -- 172.21.15.26:0/605620616 shutdown_connections 2024-04-21T10:22:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.105+0000 7f27eab4f700 1 --2- 172.21.15.26:0/605620616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e41015f0 0x7f27e41019d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.105+0000 7f27eab4f700 1 -- 172.21.15.26:0/605620616 >> 172.21.15.26:0/605620616 conn(0x7f27e4076ea0 msgr2=0x7f27e40772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.105+0000 7f27eab4f700 1 -- 172.21.15.26:0/605620616 shutdown_connections 2024-04-21T10:22:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.105+0000 7f27eab4f700 1 -- 172.21.15.26:0/605620616 wait complete. 2024-04-21T10:22:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.106+0000 7f27eab4f700 1 Processor -- start 2024-04-21T10:22:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.106+0000 7f27eab4f700 1 -- start start 2024-04-21T10:22:28.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.106+0000 7f27eab4f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e41015f0 0x7f27e4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:28.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.106+0000 7f27eab4f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f27e410b820 con 0x7f27e41015f0 2024-04-21T10:22:28.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.106+0000 7f27e88eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e41015f0 0x7f27e4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:28.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.106+0000 7f27e88eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e41015f0 0x7f27e4197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54262/0 (socket says 172.21.15.26:54262) 2024-04-21T10:22:28.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.106+0000 7f27e88eb700 1 -- 172.21.15.26:0/1842036975 learned_addr learned my addr 172.21.15.26:0/1842036975 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:28.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.106+0000 7f27e88eb700 1 -- 172.21.15.26:0/1842036975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e41983c0 con 0x7f27e41015f0 2024-04-21T10:22:28.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.106+0000 7f27e88eb700 1 --2- 172.21.15.26:0/1842036975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e41015f0 0x7f27e4197e80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f27d4006010 tx=0x7f27d4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:28.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.107+0000 7f27e1ffb700 1 -- 172.21.15.26:0/1842036975 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d4026070 con 0x7f27e41015f0 2024-04-21T10:22:28.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.107+0000 7f27e1ffb700 1 -- 172.21.15.26:0/1842036975 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d400bba0 con 0x7f27e41015f0 2024-04-21T10:22:28.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.107+0000 7f27eab4f700 1 -- 172.21.15.26:0/1842036975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e41986b0 con 0x7f27e41015f0 2024-04-21T10:22:28.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.107+0000 7f27e1ffb700 1 -- 172.21.15.26:0/1842036975 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d4003910 con 0x7f27e41015f0 2024-04-21T10:22:28.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.107+0000 7f27eab4f700 1 -- 172.21.15.26:0/1842036975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e419c750 con 0x7f27e41015f0 2024-04-21T10:22:28.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.108+0000 7f27e1ffb700 1 -- 172.21.15.26:0/1842036975 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f27d4029070 con 0x7f27e41015f0 2024-04-21T10:22:28.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.108+0000 7f27eab4f700 1 -- 172.21.15.26:0/1842036975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e4102a80 con 0x7f27e41015f0 2024-04-21T10:22:28.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.108+0000 7f27e1ffb700 1 --2- 172.21.15.26:0/1842036975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f27cc0396b0 0x7f27cc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:28.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.108+0000 7f27e1ffb700 1 -- 172.21.15.26:0/1842036975 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27d401e070 con 0x7f27e41015f0 2024-04-21T10:22:28.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.108+0000 7f27e3fff700 1 --2- 172.21.15.26:0/1842036975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f27cc0396b0 0x7f27cc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:28.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.109+0000 7f27e3fff700 1 --2- 172.21.15.26:0/1842036975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f27cc0396b0 0x7f27cc03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f27d8006fd0 tx=0x7f27d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:28.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.113+0000 7f27e1ffb700 1 -- 172.21.15.26:0/1842036975 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27d4013070 con 0x7f27e41015f0 2024-04-21T10:22:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:28.326+0000 7f27eab4f700 1 -- 172.21.15.26:0/1842036975 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}) -- 0x7f27e41991e0 con 0x7f27cc0396b0 2024-04-21T10:22:30.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:29 smithi026 ceph-mon[27965]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:30.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:29 smithi026 ceph-mon[27965]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:22:30.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:30.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:30.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:30.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:30.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:22:30.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:30.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:22:30.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:29 smithi026 ceph-mon[27965]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-21T10:22:31.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:30 smithi026 ceph-mon[27965]: Deploying cephadm binary to smithi190 2024-04-21T10:22:31.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:30 smithi026 ceph-mon[27965]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:22:31.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:30 smithi026 ceph-mon[27965]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:22:31.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:30 smithi026 ceph-mon[27965]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:22:32.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:31 smithi026 ceph-mon[27965]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:32.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:32.578+0000 7f27e1ffb700 1 -- 172.21.15.26:0/1842036975 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f27e41991e0 con 0x7f27cc0396b0 2024-04-21T10:22:32.579 INFO:teuthology.orchestra.run.smithi026.stdout:Added host 'smithi190' with addr '172.21.15.190' 2024-04-21T10:22:32.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:32.581+0000 7f27eab4f700 1 -- 172.21.15.26:0/1842036975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f27cc0396b0 msgr2=0x7f27cc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:32.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:32.581+0000 7f27eab4f700 1 --2- 172.21.15.26:0/1842036975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f27cc0396b0 0x7f27cc03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f27d8006fd0 tx=0x7f27d8006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:32.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:32.581+0000 7f27eab4f700 1 -- 172.21.15.26:0/1842036975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e41015f0 msgr2=0x7f27e4197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:32.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:32.581+0000 7f27eab4f700 1 --2- 172.21.15.26:0/1842036975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e41015f0 0x7f27e4197e80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f27d4006010 tx=0x7f27d4005780 comp rx=0 tx=0).stop 2024-04-21T10:22:32.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:32.581+0000 7f27eab4f700 1 -- 172.21.15.26:0/1842036975 shutdown_connections 2024-04-21T10:22:32.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:32.581+0000 7f27eab4f700 1 --2- 172.21.15.26:0/1842036975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f27cc0396b0 0x7f27cc03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:32.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:32.581+0000 7f27eab4f700 1 --2- 172.21.15.26:0/1842036975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27e41015f0 0x7f27e4197e80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:32.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:32.581+0000 7f27eab4f700 1 -- 172.21.15.26:0/1842036975 >> 172.21.15.26:0/1842036975 conn(0x7f27e4076ea0 msgr2=0x7f27e4069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:32.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:32.581+0000 7f27eab4f700 1 -- 172.21.15.26:0/1842036975 shutdown_connections 2024-04-21T10:22:32.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:32.581+0000 7f27eab4f700 1 -- 172.21.15.26:0/1842036975 wait complete. 2024-04-21T10:22:33.174 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:33.174 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:33.174 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:33.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:33 smithi026 ceph-mon[27965]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:33.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:33.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:33 smithi026 ceph-mon[27965]: Added host smithi190 2024-04-21T10:22:33.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:22:33.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-21T10:22:33.449 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:22:34.404 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:34.404 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:34.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.880+0000 7f67929c5700 1 -- 172.21.15.26:0/991998794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c103650 msgr2=0x7f678c103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.880+0000 7f67929c5700 1 --2- 172.21.15.26:0/991998794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c103650 0x7f678c103a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f677c009910 tx=0x7f677c009c20 comp rx=0 tx=0).stop 2024-04-21T10:22:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.880+0000 7f67929c5700 1 -- 172.21.15.26:0/991998794 shutdown_connections 2024-04-21T10:22:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.880+0000 7f67929c5700 1 --2- 172.21.15.26:0/991998794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c103650 0x7f678c103a30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.880+0000 7f67929c5700 1 -- 172.21.15.26:0/991998794 >> 172.21.15.26:0/991998794 conn(0x7f678c0fcea0 msgr2=0x7f678c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.880+0000 7f67929c5700 1 -- 172.21.15.26:0/991998794 shutdown_connections 2024-04-21T10:22:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.880+0000 7f67929c5700 1 -- 172.21.15.26:0/991998794 wait complete. 2024-04-21T10:22:34.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.881+0000 7f67929c5700 1 Processor -- start 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.881+0000 7f67929c5700 1 -- start start 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.881+0000 7f67929c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c103650 0x7f678c1012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.881+0000 7f67929c5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f678c10c9d0 con 0x7f678c103650 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.882+0000 7f678bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c103650 0x7f678c1012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.882+0000 7f678bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c103650 0x7f678c1012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36992/0 (socket says 172.21.15.26:36992) 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.882+0000 7f678bfff700 1 -- 172.21.15.26:0/1196162352 learned_addr learned my addr 172.21.15.26:0/1196162352 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.882+0000 7f678bfff700 1 -- 172.21.15.26:0/1196162352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c1031a0 con 0x7f678c103650 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.882+0000 7f678bfff700 1 --2- 172.21.15.26:0/1196162352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c103650 0x7f678c1012b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f677c0056d0 tx=0x7f677c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.882+0000 7f67897fa700 1 -- 172.21.15.26:0/1196162352 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f677c026070 con 0x7f678c103650 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.882+0000 7f67897fa700 1 -- 172.21.15.26:0/1196162352 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f677c004370 con 0x7f678c103650 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.882+0000 7f67929c5700 1 -- 172.21.15.26:0/1196162352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678c1017f0 con 0x7f678c103650 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.882+0000 7f67897fa700 1 -- 172.21.15.26:0/1196162352 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f677c0039b0 con 0x7f678c103650 2024-04-21T10:22:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.882+0000 7f67929c5700 1 -- 172.21.15.26:0/1196162352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678c101cb0 con 0x7f678c103650 2024-04-21T10:22:34.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.883+0000 7f67897fa700 1 -- 172.21.15.26:0/1196162352 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f677c029070 con 0x7f678c103650 2024-04-21T10:22:34.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.883+0000 7f67929c5700 1 -- 172.21.15.26:0/1196162352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f678c06b190 con 0x7f678c103650 2024-04-21T10:22:34.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.883+0000 7f67897fa700 1 --2- 172.21.15.26:0/1196162352 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f67740396b0 0x7f677403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:34.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.883+0000 7f67897fa700 1 -- 172.21.15.26:0/1196162352 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f677c01e070 con 0x7f678c103650 2024-04-21T10:22:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.883+0000 7f678b7fe700 1 --2- 172.21.15.26:0/1196162352 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f67740396b0 0x7f677403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.884+0000 7f678b7fe700 1 --2- 172.21.15.26:0/1196162352 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f67740396b0 0x7f677403bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6780006fd0 tx=0x7f6780006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:34.888+0000 7f67897fa700 1 -- 172.21.15.26:0/1196162352 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f677c013070 con 0x7f678c103650 2024-04-21T10:22:35.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:35.105+0000 7f67929c5700 1 -- 172.21.15.26:0/1196162352 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f678c1027e0 con 0x7f67740396b0 2024-04-21T10:22:35.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:35.106+0000 7f67897fa700 1 -- 172.21.15.26:0/1196162352 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f678c1027e0 con 0x7f67740396b0 2024-04-21T10:22:35.106 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:22:35.106 INFO:teuthology.orchestra.run.smithi026.stdout:[{"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.169", "hostname": "smithi169", "labels": [], "status": ""}, {"addr": "172.21.15.190", "hostname": "smithi190", "labels": [], "status": ""}] 2024-04-21T10:22:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:35.108+0000 7f67929c5700 1 -- 172.21.15.26:0/1196162352 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f67740396b0 msgr2=0x7f677403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:35.108+0000 7f67929c5700 1 --2- 172.21.15.26:0/1196162352 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f67740396b0 0x7f677403bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6780006fd0 tx=0x7f6780006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:35.108+0000 7f67929c5700 1 -- 172.21.15.26:0/1196162352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c103650 msgr2=0x7f678c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:35.108+0000 7f67929c5700 1 --2- 172.21.15.26:0/1196162352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c103650 0x7f678c1012b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f677c0056d0 tx=0x7f677c0057b0 comp rx=0 tx=0).stop 2024-04-21T10:22:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:35.108+0000 7f67929c5700 1 -- 172.21.15.26:0/1196162352 shutdown_connections 2024-04-21T10:22:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:35.108+0000 7f67929c5700 1 --2- 172.21.15.26:0/1196162352 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f67740396b0 0x7f677403bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:35.108+0000 7f67929c5700 1 --2- 172.21.15.26:0/1196162352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678c103650 0x7f678c1012b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:35.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:35.108+0000 7f67929c5700 1 -- 172.21.15.26:0/1196162352 >> 172.21.15.26:0/1196162352 conn(0x7f678c0fcea0 msgr2=0x7f678c1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:35.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:35.108+0000 7f67929c5700 1 -- 172.21.15.26:0/1196162352 shutdown_connections 2024-04-21T10:22:35.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:35.108+0000 7f67929c5700 1 -- 172.21.15.26:0/1196162352 wait complete. 2024-04-21T10:22:35.683 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-21T10:22:35.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-21T10:22:35.924 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:22:35.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:35 smithi026 ceph-mon[27965]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:35.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:35 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:36.885 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:36 smithi026 ceph-mon[27965]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:22:37.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.353+0000 7ff66beec700 1 -- 172.21.15.26:0/158875273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6641084e0 msgr2=0x7ff6641088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:37.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.353+0000 7ff66beec700 1 --2- 172.21.15.26:0/158875273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6641084e0 0x7ff6641088c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff654009910 tx=0x7ff654009c20 comp rx=0 tx=0).stop 2024-04-21T10:22:37.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.353+0000 7ff66beec700 1 -- 172.21.15.26:0/158875273 shutdown_connections 2024-04-21T10:22:37.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.353+0000 7ff66beec700 1 --2- 172.21.15.26:0/158875273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6641084e0 0x7ff6641088c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:37.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.353+0000 7ff66beec700 1 -- 172.21.15.26:0/158875273 >> 172.21.15.26:0/158875273 conn(0x7ff664076ea0 msgr2=0x7ff6640772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:37.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.353+0000 7ff66beec700 1 -- 172.21.15.26:0/158875273 shutdown_connections 2024-04-21T10:22:37.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.353+0000 7ff66beec700 1 -- 172.21.15.26:0/158875273 wait complete. 2024-04-21T10:22:37.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.354+0000 7ff66beec700 1 Processor -- start 2024-04-21T10:22:37.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.354+0000 7ff66beec700 1 -- start start 2024-04-21T10:22:37.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.354+0000 7ff66beec700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6641084e0 0x7ff664197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:37.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.354+0000 7ff66beec700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff66410c920 con 0x7ff6641084e0 2024-04-21T10:22:37.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.354+0000 7ff669c88700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6641084e0 0x7ff664197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:37.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.354+0000 7ff669c88700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6641084e0 0x7ff664197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37012/0 (socket says 172.21.15.26:37012) 2024-04-21T10:22:37.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.354+0000 7ff669c88700 1 -- 172.21.15.26:0/45354162 learned_addr learned my addr 172.21.15.26:0/45354162 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:22:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.354+0000 7ff669c88700 1 -- 172.21.15.26:0/45354162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff664198400 con 0x7ff6641084e0 2024-04-21T10:22:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.355+0000 7ff669c88700 1 --2- 172.21.15.26:0/45354162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6641084e0 0x7ff664197ec0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff6540056d0 tx=0x7ff6540057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.355+0000 7ff65affd700 1 -- 172.21.15.26:0/45354162 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff654026070 con 0x7ff6641084e0 2024-04-21T10:22:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.355+0000 7ff65affd700 1 -- 172.21.15.26:0/45354162 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff65400bba0 con 0x7ff6641084e0 2024-04-21T10:22:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.355+0000 7ff66beec700 1 -- 172.21.15.26:0/45354162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6641986f0 con 0x7ff6641084e0 2024-04-21T10:22:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.355+0000 7ff65affd700 1 -- 172.21.15.26:0/45354162 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff654003920 con 0x7ff6641084e0 2024-04-21T10:22:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.355+0000 7ff66beec700 1 -- 172.21.15.26:0/45354162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66419c790 con 0x7ff6641084e0 2024-04-21T10:22:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.356+0000 7ff65affd700 1 -- 172.21.15.26:0/45354162 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff654029070 con 0x7ff6641084e0 2024-04-21T10:22:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.356+0000 7ff65affd700 1 --2- 172.21.15.26:0/45354162 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff6500396b0 0x7ff65003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.356+0000 7ff65affd700 1 -- 172.21.15.26:0/45354162 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff65401e070 con 0x7ff6641084e0 2024-04-21T10:22:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.356+0000 7ff669487700 1 --2- 172.21.15.26:0/45354162 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff6500396b0 0x7ff65003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.356+0000 7ff66beec700 1 -- 172.21.15.26:0/45354162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff664103b90 con 0x7ff6641084e0 2024-04-21T10:22:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.357+0000 7ff669487700 1 --2- 172.21.15.26:0/45354162 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff6500396b0 0x7ff65003bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff660006fd0 tx=0x7ff660006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:37.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.361+0000 7ff65affd700 1 -- 172.21.15.26:0/45354162 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff654013070 con 0x7ff6641084e0 2024-04-21T10:22:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.575+0000 7ff66beec700 1 -- 172.21.15.26:0/45354162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff664199220 con 0x7ff6641084e0 2024-04-21T10:22:37.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.635+0000 7ff65affd700 1 -- 172.21.15.26:0/45354162 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7ff654003db0 con 0x7ff6641084e0 2024-04-21T10:22:37.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.639+0000 7ff66beec700 1 -- 172.21.15.26:0/45354162 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff6500396b0 msgr2=0x7ff65003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:37.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.639+0000 7ff66beec700 1 --2- 172.21.15.26:0/45354162 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff6500396b0 0x7ff65003bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff660006fd0 tx=0x7ff660006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:37.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.639+0000 7ff66beec700 1 -- 172.21.15.26:0/45354162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6641084e0 msgr2=0x7ff664197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:37.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.639+0000 7ff66beec700 1 --2- 172.21.15.26:0/45354162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6641084e0 0x7ff664197ec0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff6540056d0 tx=0x7ff6540057b0 comp rx=0 tx=0).stop 2024-04-21T10:22:37.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.639+0000 7ff66beec700 1 -- 172.21.15.26:0/45354162 shutdown_connections 2024-04-21T10:22:37.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.639+0000 7ff66beec700 1 --2- 172.21.15.26:0/45354162 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff6500396b0 0x7ff65003bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:37.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.639+0000 7ff66beec700 1 --2- 172.21.15.26:0/45354162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6641084e0 0x7ff664197ec0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:37.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.639+0000 7ff66beec700 1 -- 172.21.15.26:0/45354162 >> 172.21.15.26:0/45354162 conn(0x7ff664076ea0 msgr2=0x7ff6640fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:37.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.639+0000 7ff66beec700 1 -- 172.21.15.26:0/45354162 shutdown_connections 2024-04-21T10:22:37.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:22:37.639+0000 7ff66beec700 1 -- 172.21.15.26:0/45354162 wait complete. 2024-04-21T10:22:37.640 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2024-04-21T10:22:37.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:37 smithi026 ceph-mon[27965]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:37.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:37 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/45354162' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-21T10:22:38.208 INFO:tasks.cephadm:Adding mon.a on smithi026 2024-04-21T10:22:38.208 INFO:tasks.cephadm:Adding mon.b on smithi169 2024-04-21T10:22:38.208 INFO:tasks.cephadm:Adding mon.c on smithi190 2024-04-21T10:22:38.208 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch apply mon '3;smithi026:172.21.15.26=a;smithi169:172.21.15.169=b;smithi190:172.21.15.190=c' 2024-04-21T10:22:38.459 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-21T10:22:38.507 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-21T10:22:38.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:38 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/45354162' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-21T10:22:38.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:38 smithi026 ceph-mon[27965]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T10:22:38.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:38.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:38.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:38.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:38.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:22:38.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:38.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:22:39.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:39 smithi026 ceph-mon[27965]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:39.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:39 smithi026 ceph-mon[27965]: Updating smithi190:/etc/ceph/ceph.conf 2024-04-21T10:22:39.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:39 smithi026 ceph-mon[27965]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:22:39.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:39 smithi026 ceph-mon[27965]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:22:39.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:39 smithi026 ceph-mon[27965]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:22:39.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:39 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:39.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:39 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:39.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:39 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:40.090 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.088+0000 7fbe5d503700 1 -- 172.21.15.190:0/1921600626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe58108550 msgr2=0x7fbe58108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:40.090 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.088+0000 7fbe5d503700 1 --2- 172.21.15.190:0/1921600626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe58108550 0x7fbe58108930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009910 tx=0x7fbe48009c20 comp rx=0 tx=0).stop 2024-04-21T10:22:40.090 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.088+0000 7fbe5d503700 1 -- 172.21.15.190:0/1921600626 shutdown_connections 2024-04-21T10:22:40.090 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.088+0000 7fbe5d503700 1 --2- 172.21.15.190:0/1921600626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe58108550 0x7fbe58108930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:40.090 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.088+0000 7fbe5d503700 1 -- 172.21.15.190:0/1921600626 >> 172.21.15.190:0/1921600626 conn(0x7fbe580fdf90 msgr2=0x7fbe581003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:40.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.088+0000 7fbe5d503700 1 -- 172.21.15.190:0/1921600626 shutdown_connections 2024-04-21T10:22:40.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.088+0000 7fbe5d503700 1 -- 172.21.15.190:0/1921600626 wait complete. 2024-04-21T10:22:40.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.089+0000 7fbe5d503700 1 Processor -- start 2024-04-21T10:22:40.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.089+0000 7fbe5d503700 1 -- start start 2024-04-21T10:22:40.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.089+0000 7fbe5d503700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe58108550 0x7fbe58197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:40.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.089+0000 7fbe5d503700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5810c9c0 con 0x7fbe58108550 2024-04-21T10:22:40.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.090+0000 7fbe56ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe58108550 0x7fbe58197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:40.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.090+0000 7fbe56ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe58108550 0x7fbe58197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.190:46830/0 (socket says 172.21.15.190:46830) 2024-04-21T10:22:40.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.090+0000 7fbe56ffd700 1 -- 172.21.15.190:0/4020286054 learned_addr learned my addr 172.21.15.190:0/4020286054 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-21T10:22:40.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.090+0000 7fbe56ffd700 1 -- 172.21.15.190:0/4020286054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe58198400 con 0x7fbe58108550 2024-04-21T10:22:40.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.090+0000 7fbe56ffd700 1 --2- 172.21.15.190:0/4020286054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe58108550 0x7fbe58197ec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbe480056d0 tx=0x7fbe480057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:40.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.090+0000 7fbe54ff9700 1 -- 172.21.15.190:0/4020286054 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe48026070 con 0x7fbe58108550 2024-04-21T10:22:40.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.090+0000 7fbe5d503700 1 -- 172.21.15.190:0/4020286054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe58198690 con 0x7fbe58108550 2024-04-21T10:22:40.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.090+0000 7fbe54ff9700 1 -- 172.21.15.190:0/4020286054 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4800bc00 con 0x7fbe58108550 2024-04-21T10:22:40.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.091+0000 7fbe54ff9700 1 -- 172.21.15.190:0/4020286054 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe48003920 con 0x7fbe58108550 2024-04-21T10:22:40.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.091+0000 7fbe5d503700 1 -- 172.21.15.190:0/4020286054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe58198af0 con 0x7fbe58108550 2024-04-21T10:22:40.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.091+0000 7fbe5d503700 1 -- 172.21.15.190:0/4020286054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe58103bc0 con 0x7fbe58108550 2024-04-21T10:22:40.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.091+0000 7fbe54ff9700 1 -- 172.21.15.190:0/4020286054 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbe48029070 con 0x7fbe58108550 2024-04-21T10:22:40.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.092+0000 7fbe54ff9700 1 --2- 172.21.15.190:0/4020286054 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbe44039650 0x7fbe4403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:40.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.092+0000 7fbe54ff9700 1 -- 172.21.15.190:0/4020286054 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbe4801e070 con 0x7fbe58108550 2024-04-21T10:22:40.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.092+0000 7fbe4e7fc700 1 --2- 172.21.15.190:0/4020286054 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbe44039650 0x7fbe4403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:40.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.092+0000 7fbe4e7fc700 1 --2- 172.21.15.190:0/4020286054 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbe44039650 0x7fbe4403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbe40006fd0 tx=0x7fbe40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:40.097 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.097+0000 7fbe54ff9700 1 -- 172.21.15.190:0/4020286054 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe48013070 con 0x7fbe58108550 2024-04-21T10:22:40.307 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.307+0000 7fbe5d503700 1 -- 172.21.15.190:0/4020286054 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi169:172.21.15.169=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}) -- 0x7fbe580fea80 con 0x7fbe44039650 2024-04-21T10:22:40.314 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.314+0000 7fbe54ff9700 1 -- 172.21.15.190:0/4020286054 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbe580fea80 con 0x7fbe44039650 2024-04-21T10:22:40.314 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mon update... 2024-04-21T10:22:40.316 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.316+0000 7fbe5d503700 1 -- 172.21.15.190:0/4020286054 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbe44039650 msgr2=0x7fbe4403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:40.316 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.316+0000 7fbe5d503700 1 --2- 172.21.15.190:0/4020286054 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbe44039650 0x7fbe4403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbe40006fd0 tx=0x7fbe40006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:40.316 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.316+0000 7fbe5d503700 1 -- 172.21.15.190:0/4020286054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe58108550 msgr2=0x7fbe58197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:40.316 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.316+0000 7fbe5d503700 1 --2- 172.21.15.190:0/4020286054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe58108550 0x7fbe58197ec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbe480056d0 tx=0x7fbe480057b0 comp rx=0 tx=0).stop 2024-04-21T10:22:40.316 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.316+0000 7fbe5d503700 1 -- 172.21.15.190:0/4020286054 shutdown_connections 2024-04-21T10:22:40.316 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.316+0000 7fbe5d503700 1 --2- 172.21.15.190:0/4020286054 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbe44039650 0x7fbe4403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:40.317 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.316+0000 7fbe5d503700 1 --2- 172.21.15.190:0/4020286054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe58108550 0x7fbe58197ec0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:40.317 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.316+0000 7fbe5d503700 1 -- 172.21.15.190:0/4020286054 >> 172.21.15.190:0/4020286054 conn(0x7fbe580fdf90 msgr2=0x7fbe580fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:40.317 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.317+0000 7fbe5d503700 1 -- 172.21.15.190:0/4020286054 shutdown_connections 2024-04-21T10:22:40.317 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:40.317+0000 7fbe5d503700 1 -- 172.21.15.190:0/4020286054 wait complete. 2024-04-21T10:22:40.907 DEBUG:teuthology.orchestra.run.smithi169:mon.b> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.b.service 2024-04-21T10:22:40.909 DEBUG:teuthology.orchestra.run.smithi190:mon.c> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.c.service 2024-04-21T10:22:40.912 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T10:22:40.912 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T10:22:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:41 smithi026 ceph-mon[27965]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:41 smithi026 ceph-mon[27965]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi169:172.21.15.169=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:22:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:41 smithi026 ceph-mon[27965]: Saving service mon spec with placement smithi026:172.21.15.26=a;smithi169:172.21.15.169=b;smithi190:172.21.15.190=c;count:3 2024-04-21T10:22:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:22:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:41.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:22:41.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:41.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:22:41.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:41.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:41 smithi026 ceph-mon[27965]: Deploying daemon mon.c on smithi190 2024-04-21T10:22:42.728 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:22:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:43 smithi026 ceph-mon[27965]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:44.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:43 smithi190 systemd[1]: Starting Ceph mon.c for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:22:44.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 podman[31377]: 2024-04-21 10:22:44.246356878 +0000 UTC m=+0.147687955 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:22:44.732 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 podman[31377]: 2024-04-21 10:22:44.372130424 +0000 UTC m=+0.273461534 container create 5f7e1da45384ff7fdf5aba43c50caaf7a28ea11189e577a554aa9180001c3a96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2) 2024-04-21T10:22:44.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.832+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/821219039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6e81074c0 msgr2=0x7fa6e81078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:44.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.832+0000 7fa6ef4b3700 1 --2- 172.21.15.190:0/821219039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6e81074c0 0x7fa6e81078a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009910 tx=0x7fa6e0009c20 comp rx=0 tx=0).stop 2024-04-21T10:22:44.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.832+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/821219039 shutdown_connections 2024-04-21T10:22:44.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.832+0000 7fa6ef4b3700 1 --2- 172.21.15.190:0/821219039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6e81074c0 0x7fa6e81078a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:44.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.832+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/821219039 >> 172.21.15.190:0/821219039 conn(0x7fa6e80fcf20 msgr2=0x7fa6e80ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:44.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.832+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/821219039 shutdown_connections 2024-04-21T10:22:44.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.832+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/821219039 wait complete. 2024-04-21T10:22:44.835 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.833+0000 7fa6ef4b3700 1 Processor -- start 2024-04-21T10:22:44.835 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.833+0000 7fa6ef4b3700 1 -- start start 2024-04-21T10:22:44.835 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.833+0000 7fa6ef4b3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6e81074c0 0x7fa6e8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:44.835 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.833+0000 7fa6ef4b3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e810b920 con 0x7fa6e81074c0 2024-04-21T10:22:44.836 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.834+0000 7fa6ed24f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6e81074c0 0x7fa6e8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:44.836 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.834+0000 7fa6ed24f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6e81074c0 0x7fa6e8197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.190:46854/0 (socket says 172.21.15.190:46854) 2024-04-21T10:22:44.836 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.834+0000 7fa6ed24f700 1 -- 172.21.15.190:0/509684 learned_addr learned my addr 172.21.15.190:0/509684 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-21T10:22:44.836 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.834+0000 7fa6ed24f700 1 -- 172.21.15.190:0/509684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e8198510 con 0x7fa6e81074c0 2024-04-21T10:22:44.836 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.834+0000 7fa6ed24f700 1 --2- 172.21.15.190:0/509684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6e81074c0 0x7fa6e8197fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00056d0 tx=0x7fa6e00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:44.836 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.834+0000 7fa6e67fc700 1 -- 172.21.15.190:0/509684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6e0026070 con 0x7fa6e81074c0 2024-04-21T10:22:44.836 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.835+0000 7fa6e67fc700 1 -- 172.21.15.190:0/509684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6e000bba0 con 0x7fa6e81074c0 2024-04-21T10:22:44.836 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.835+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/509684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e8198800 con 0x7fa6e81074c0 2024-04-21T10:22:44.836 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.835+0000 7fa6e67fc700 1 -- 172.21.15.190:0/509684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6e0003920 con 0x7fa6e81074c0 2024-04-21T10:22:44.836 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.835+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/509684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e819c8a0 con 0x7fa6e81074c0 2024-04-21T10:22:44.837 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.835+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/509684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e8102b70 con 0x7fa6e81074c0 2024-04-21T10:22:44.837 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.836+0000 7fa6e67fc700 1 -- 172.21.15.190:0/509684 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa6e0029070 con 0x7fa6e81074c0 2024-04-21T10:22:44.837 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.836+0000 7fa6e67fc700 1 --2- 172.21.15.190:0/509684 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa6d00396b0 0x7fa6d003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:44.837 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.836+0000 7fa6e67fc700 1 -- 172.21.15.190:0/509684 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa6e001e070 con 0x7fa6e81074c0 2024-04-21T10:22:44.837 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.836+0000 7fa6eca4e700 1 --2- 172.21.15.190:0/509684 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa6d00396b0 0x7fa6d003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:44.837 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.836+0000 7fa6eca4e700 1 --2- 172.21.15.190:0/509684 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa6d00396b0 0x7fa6d003bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc006fd0 tx=0x7fa6dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:44.842 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.841+0000 7fa6e67fc700 1 -- 172.21.15.190:0/509684 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6e0013070 con 0x7fa6e81074c0 2024-04-21T10:22:44.967 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:44.966+0000 7fa6e67fc700 1 -- 172.21.15.190:0/509684 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa6e0029360 con 0x7fa6e81074c0 2024-04-21T10:22:44.982 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 podman[31377]: 2024-04-21 10:22:44.909094208 +0000 UTC m=+0.810425286 container init 5f7e1da45384ff7fdf5aba43c50caaf7a28ea11189e577a554aa9180001c3a96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-21T10:22:44.982 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 podman[31377]: 2024-04-21 10:22:44.914063988 +0000 UTC m=+0.815395064 container start 5f7e1da45384ff7fdf5aba43c50caaf7a28ea11189e577a554aa9180001c3a96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-c, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416) 2024-04-21T10:22:44.982 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: set uid:gid to 167:167 (ceph:ceph) 2024-04-21T10:22:44.982 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev), process ceph-mon, pid 2 2024-04-21T10:22:44.982 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: pidfile_write: ignore empty --pid-file 2024-04-21T10:22:44.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: load: jerasure load: lrc 2024-04-21T10:22:44.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: RocksDB version: 7.9.2 2024-04-21T10:22:44.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Git sha 0 2024-04-21T10:22:44.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Compile date 2024-04-19 14:19:22 2024-04-21T10:22:44.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: DB SUMMARY 2024-04-21T10:22:44.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: DB Session ID: G6GL4YON6A55ELZ2YHWW 2024-04-21T10:22:44.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: CURRENT file: CURRENT 2024-04-21T10:22:44.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: IDENTITY file: IDENTITY 2024-04-21T10:22:44.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-21T10:22:44.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-21T10:22:44.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-21T10:22:44.983 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.error_if_exists: 0 2024-04-21T10:22:44.984 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.create_if_missing: 0 2024-04-21T10:22:44.984 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.paranoid_checks: 1 2024-04-21T10:22:44.984 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-21T10:22:44.984 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-21T10:22:44.984 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-21T10:22:44.984 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.env: 0x55ed5fb1dcc0 2024-04-21T10:22:44.984 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.fs: PosixFileSystem 2024-04-21T10:22:44.984 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.info_log: 0x55ed60ce5be0 2024-04-21T10:22:44.984 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_file_opening_threads: 16 2024-04-21T10:22:44.985 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.statistics: (nil) 2024-04-21T10:22:44.985 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.use_fsync: 0 2024-04-21T10:22:44.985 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_log_file_size: 0 2024-04-21T10:22:44.985 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-21T10:22:44.985 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.keep_log_file_num: 1000 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.recycle_log_file_num: 0 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.allow_fallocate: 1 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.allow_mmap_reads: 0 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.allow_mmap_writes: 0 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.use_direct_reads: 0 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.create_missing_column_families: 0 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.db_log_dir: 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.wal_dir: 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-21T10:22:44.986 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.advise_random_on_open: 1 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.db_write_buffer_size: 0 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.write_buffer_manager: 0x55ed60c790e0 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.rate_limiter: (nil) 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.wal_recovery_mode: 2 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.enable_thread_tracking: 0 2024-04-21T10:22:44.987 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.enable_pipelined_write: 0 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.unordered_write: 0 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.row_cache: None 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.wal_filter: None 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.allow_ingest_behind: 0 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.two_write_queues: 0 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.manual_wal_flush: 0 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.wal_compression: 0 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.atomic_flush: 0 2024-04-21T10:22:44.988 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.log_readahead_size: 0 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.best_efforts_recovery: 0 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.allow_data_in_errors: 0 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.db_host_id: __hostname__ 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_background_jobs: 2 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_background_compactions: -1 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_subcompactions: 1 2024-04-21T10:22:44.989 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_total_wal_size: 0 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_open_files: -1 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bytes_per_sync: 0 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_readahead_size: 0 2024-04-21T10:22:44.990 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_background_flushes: -1 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Compression algorithms supported: 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: kZSTD supported: 0 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: kXpressCompression supported: 0 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: kLZ4HCCompression supported: 1 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: kZlibCompression supported: 1 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: kSnappyCompression supported: 1 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: kLZ4Compression supported: 1 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: kBZip2Compression supported: 0 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-21T10:22:44.991 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.merge_operator: 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_filter: None 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_filter_factory: None 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.sst_partitioner_factory: None 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ed60ce5dc0) 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout: cache_index_and_filter_blocks: 1 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout: pin_top_level_index_and_filter: 1 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_type: 0 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout: data_block_index_type: 0 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_shortening: 1 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout: checksum: 4 2024-04-21T10:22:44.992 INFO:journalctl@ceph.mon.c.smithi190.stdout: no_block_cache: 0 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache: 0x55ed60cab1f0 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_name: BinnedLRUCache 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_options: 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: capacity : 536870912 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: num_shard_bits : 4 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: strict_capacity_limit : 0 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: high_pri_pool_ratio: 0.000 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_compressed: (nil) 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: persistent_cache: (nil) 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_size: 4096 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_size_deviation: 10 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_restart_interval: 16 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_block_restart_interval: 1 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: metadata_block_size: 4096 2024-04-21T10:22:44.993 INFO:journalctl@ceph.mon.c.smithi190.stdout: partition_filters: 0 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout: use_delta_encoding: 1 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout: filter_policy: bloomfilter 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout: whole_key_filtering: 1 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout: verify_compression: 0 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout: read_amp_bytes_per_bit: 0 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout: format_version: 5 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_index_compression: 1 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_align: 0 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout: max_auto_readahead_size: 262144 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout: prepopulate_block_cache: 0 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout: initial_auto_readahead_size: 8192 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout: num_file_reads_for_auto_readahead: 2 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.write_buffer_size: 33554432 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_write_buffer_number: 2 2024-04-21T10:22:44.994 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compression: NoCompression 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bottommost_compression: Disabled 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.prefix_extractor: nullptr 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.num_levels: 7 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-21T10:22:44.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compression_opts.level: 32767 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compression_opts.strategy: 0 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compression_opts.enabled: false 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.target_file_size_base: 67108864 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-21T10:22:44.996 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.arena_block_size: 1048576 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-21T10:22:44.997 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.disable_auto_compactions: 0 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.inplace_update_support: 0 2024-04-21T10:22:44.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-21T10:22:44.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-21T10:22:44.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-21T10:22:44.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-21T10:22:44.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.bloom_locality: 0 2024-04-21T10:22:44.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.max_successive_merges: 0 2024-04-21T10:22:44.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-21T10:22:44.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.paranoid_file_checks: 0 2024-04-21T10:22:44.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.force_consistency_checks: 1 2024-04-21T10:22:44.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.report_bg_io_stats: 0 2024-04-21T10:22:44.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.ttl: 2592000 2024-04-21T10:22:44.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-21T10:22:44.999 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-21T10:22:45.000 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-21T10:22:45.000 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.enable_blob_files: false 2024-04-21T10:22:45.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.min_blob_size: 0 2024-04-21T10:22:45.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.blob_file_size: 268435456 2024-04-21T10:22:45.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-21T10:22:45.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-21T10:22:45.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-21T10:22:45.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-21T10:22:45.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-21T10:22:45.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.blob_file_starting_level: 0 2024-04-21T10:22:45.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-21T10:22:45.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-21T10:22:45.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-21T10:22:45.001 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ab5a6cd0-3fd1-49f0-812f-8a0b36491fc4 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713694964944047, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713694964944463, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713694964, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ab5a6cd0-3fd1-49f0-812f-8a0b36491fc4", "db_session_id": "G6GL4YON6A55ELZ2YHWW", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713694964944543, "job": 1, "event": "recovery_finished"} 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ed60dc0000 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: DB pointer 0x55ed60daa000 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** DB Stats ** 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T10:22:45.002 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** Compaction Stats [default] ** 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T10:22:45.003 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** Compaction Stats [default] ** 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-21T10:22:45.004 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout: Block cache BinnedLRUCache@0x55ed60cab1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: using public_addr v2:172.21.15.190:0/0 -> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: starting mon.c rank -1 at public addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] at bind addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(???) e0 preinit fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).mds e1 new map 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout: e1 2024-04-21T10:22:45.005 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout: legacy client fscid: -1 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout: No filesystems configured 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T10:22:45.006 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: Deploying cephadm binary to smithi190 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:22:45.007 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: Added host smithi190 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:45.008 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/45354162' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/45354162' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: Updating smithi190:/etc/ceph/ceph.conf 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:22:45.009 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi169:172.21.15.169=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: Saving service mon spec with placement smithi026:172.21.15.26=a;smithi169:172.21.15.169=b;smithi190:172.21.15.190=c;count:3 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: Deploying daemon mon.c on smithi190 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:45.010 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 ceph-mon[31423]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-21T10:22:45.011 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 bash[31377]: 5f7e1da45384ff7fdf5aba43c50caaf7a28ea11189e577a554aa9180001c3a96 2024-04-21T10:22:45.011 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:44 smithi190 systemd[1]: Started Ceph mon.c for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:22:45.136 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:45.135+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/509684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa6e8199330 con 0x7fa6e81074c0 2024-04-21T10:22:49.979 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:49.979+0000 7fa6e67fc700 1 -- 172.21.15.190:0/509684 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fa6e00282c0 con 0x7fa6e81074c0 2024-04-21T10:22:49.979 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:22:49.979 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","modified":"2024-04-21T10:22:44.967557Z","created":"2024-04-21T10:21:05.761117Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-21T10:22:49.982 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:49.982+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/509684 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa6d00396b0 msgr2=0x7fa6d003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:49.982 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:49.982+0000 7fa6ef4b3700 1 --2- 172.21.15.190:0/509684 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa6d00396b0 0x7fa6d003bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc006fd0 tx=0x7fa6dc006e40 comp rx=0 tx=0).stop 2024-04-21T10:22:49.982 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:49.982+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/509684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6e81074c0 msgr2=0x7fa6e8197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:49.982 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:49.982+0000 7fa6ef4b3700 1 --2- 172.21.15.190:0/509684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6e81074c0 0x7fa6e8197fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00056d0 tx=0x7fa6e00057b0 comp rx=0 tx=0).stop 2024-04-21T10:22:49.982 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:49.982+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/509684 shutdown_connections 2024-04-21T10:22:49.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:49.982+0000 7fa6ef4b3700 1 --2- 172.21.15.190:0/509684 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa6d00396b0 0x7fa6d003bb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:49.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:49.982+0000 7fa6ef4b3700 1 --2- 172.21.15.190:0/509684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6e81074c0 0x7fa6e8197fd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:49.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:49.982+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/509684 >> 172.21.15.190:0/509684 conn(0x7fa6e80fcf20 msgr2=0x7fa6e806a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:49.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:49.983+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/509684 shutdown_connections 2024-04-21T10:22:49.983 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:49.983+0000 7fa6ef4b3700 1 -- 172.21.15.190:0/509684 wait complete. 2024-04-21T10:22:49.983 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2024-04-21T10:22:50.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:22:50.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: mon.a calling monitor election 2024-04-21T10:22:50.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:50.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='client.? 172.21.15.190:0/509684' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T10:22:50.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: mon.c calling monitor election 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: monmap epoch 2 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: last_changed 2024-04-21T10:22:44.967557+0000 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: created 2024-04-21T10:21:05.761117+0000 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: min_mon_release 19 (squid) 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: election_strategy: 1 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-04-21T10:22:50.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-04-21T10:22:50.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: fsmap 2024-04-21T10:22:50.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T10:22:50.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: mgrmap e13: x(active, since 47s) 2024-04-21T10:22:50.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: overall HEALTH_OK 2024-04-21T10:22:50.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:50.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:50.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:50.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:22:50.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:50.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:22:50.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: mon.a calling monitor election 2024-04-21T10:22:50.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:50.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='client.? 172.21.15.190:0/509684' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T10:22:50.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:50.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:50.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:50.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: mon.c calling monitor election 2024-04-21T10:22:50.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:50.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:50.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:50.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:50.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: monmap epoch 2 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: last_changed 2024-04-21T10:22:44.967557+0000 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: created 2024-04-21T10:21:05.761117+0000 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: min_mon_release 19 (squid) 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: election_strategy: 1 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: fsmap 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: mgrmap e13: x(active, since 47s) 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: overall HEALTH_OK 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:22:50.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:51.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:51 smithi026 ceph-mon[27965]: Deploying daemon mon.b on smithi169 2024-04-21T10:22:51.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:51 smithi026 ceph-mon[27965]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:51.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:51.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:51 smithi190 ceph-mon[31423]: Deploying daemon mon.b on smithi169 2024-04-21T10:22:51.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:51 smithi190 ceph-mon[31423]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:51.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:51.587 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T10:22:51.587 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T10:22:51.832 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:22:53.279 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.279+0000 7f0f04ba8700 1 -- 172.21.15.190:0/150625206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f00103710 msgr2=0x7f0f00103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:53.279 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.279+0000 7f0f04ba8700 1 --2- 172.21.15.190:0/150625206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f00103710 0x7f0f00103af0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009910 tx=0x7f0ef0009c20 comp rx=0 tx=0).stop 2024-04-21T10:22:53.280 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.280+0000 7f0f04ba8700 1 -- 172.21.15.190:0/150625206 shutdown_connections 2024-04-21T10:22:53.280 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.280+0000 7f0f04ba8700 1 --2- 172.21.15.190:0/150625206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f00103710 0x7f0f00103af0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:53.280 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.280+0000 7f0f04ba8700 1 -- 172.21.15.190:0/150625206 >> 172.21.15.190:0/150625206 conn(0x7f0f000fcf80 msgr2=0x7f0f000ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:53.280 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.280+0000 7f0f04ba8700 1 -- 172.21.15.190:0/150625206 shutdown_connections 2024-04-21T10:22:53.280 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.280+0000 7f0f04ba8700 1 -- 172.21.15.190:0/150625206 wait complete. 2024-04-21T10:22:53.282 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.280+0000 7f0f04ba8700 1 Processor -- start 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.280+0000 7f0f04ba8700 1 -- start start 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.280+0000 7f0f04ba8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f00103710 0x7f0f00198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0f04ba8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f001987a0 0x7f0f0019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0f04ba8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0010cab0 con 0x7f0f001987a0 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0f04ba8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0010c930 con 0x7f0f00103710 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0efe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f00103710 0x7f0f00198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0efe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f00103710 0x7f0f00198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:52870/0 (socket says 172.21.15.190:52870) 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0efe59c700 1 -- 172.21.15.190:0/3290581986 learned_addr learned my addr 172.21.15.190:0/3290581986 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0efe59c700 1 -- 172.21.15.190:0/3290581986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f00103710 msgr2=0x7f0f00198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0efe59c700 1 -- 172.21.15.190:0/3290581986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f00103710 msgr2=0x7f0f00198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0efe59c700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f00103710 0x7f0f00198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0efe59c700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f00103710 0x7f0f00198260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T10:22:53.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0efdd9b700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f001987a0 0x7f0f0019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0efdd9b700 1 -- 172.21.15.190:0/3290581986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f00103710 msgr2=0x7f0f00198260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0efdd9b700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f00103710 0x7f0f00198260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0efdd9b700 1 -- 172.21.15.190:0/3290581986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f00198de0 con 0x7f0f001987a0 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.281+0000 7f0efdd9b700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f001987a0 0x7f0f0019cae0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400b4d0 tx=0x7f0ef400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.282+0000 7f0eef7fe700 1 -- 172.21.15.190:0/3290581986 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0ef4011630 con 0x7f0f001987a0 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.282+0000 7f0f04ba8700 1 -- 172.21.15.190:0/3290581986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0019d020 con 0x7f0f001987a0 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.282+0000 7f0eef7fe700 1 -- 172.21.15.190:0/3290581986 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef4011c80 con 0x7f0f001987a0 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.282+0000 7f0f04ba8700 1 -- 172.21.15.190:0/3290581986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0019d580 con 0x7f0f001987a0 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.282+0000 7f0eef7fe700 1 -- 172.21.15.190:0/3290581986 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0ef4010d40 con 0x7f0f001987a0 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.283+0000 7f0f04ba8700 1 -- 172.21.15.190:0/3290581986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f0019d8f0 con 0x7f0f001987a0 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.283+0000 7f0eef7fe700 1 -- 172.21.15.190:0/3290581986 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0ef4018020 con 0x7f0f001987a0 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.283+0000 7f0eef7fe700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0ee80393f0 0x7f0ee803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.283+0000 7f0eef7fe700 1 -- 172.21.15.190:0/3290581986 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0ef404c1d0 con 0x7f0f001987a0 2024-04-21T10:22:53.284 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.283+0000 7f0efe59c700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0ee80393f0 0x7f0ee803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:53.285 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.284+0000 7f0efe59c700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0ee80393f0 0x7f0ee803b8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009750 tx=0x7f0ef001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:53.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.288+0000 7f0eef7fe700 1 -- 172.21.15.190:0/3290581986 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ef401f020 con 0x7f0f001987a0 2024-04-21T10:22:53.416 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:53 smithi190 ceph-mon[31423]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:53.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:53 smithi026 ceph-mon[27965]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:53.592 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.592+0000 7f0f04ba8700 1 -- 172.21.15.190:0/3290581986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0f00199570 con 0x7f0f001987a0 2024-04-21T10:22:53.593 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.593+0000 7f0eef7fe700 1 -- 172.21.15.190:0/3290581986 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f0ef4014170 con 0x7f0f001987a0 2024-04-21T10:22:53.593 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:22:53.593 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","modified":"2024-04-21T10:22:44.967557Z","created":"2024-04-21T10:21:05.761117Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-21T10:22:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.595+0000 7f0f04ba8700 1 -- 172.21.15.190:0/3290581986 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0ee80393f0 msgr2=0x7f0ee803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.595+0000 7f0f04ba8700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0ee80393f0 0x7f0ee803b8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009750 tx=0x7f0ef001a040 comp rx=0 tx=0).stop 2024-04-21T10:22:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.595+0000 7f0f04ba8700 1 -- 172.21.15.190:0/3290581986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f001987a0 msgr2=0x7f0f0019cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.595+0000 7f0f04ba8700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f001987a0 0x7f0f0019cae0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400b4d0 tx=0x7f0ef400b890 comp rx=0 tx=0).stop 2024-04-21T10:22:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.595+0000 7f0f04ba8700 1 -- 172.21.15.190:0/3290581986 shutdown_connections 2024-04-21T10:22:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.595+0000 7f0f04ba8700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0ee80393f0 0x7f0ee803b8b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.595+0000 7f0f04ba8700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f00103710 0x7f0f00198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.595+0000 7f0f04ba8700 1 --2- 172.21.15.190:0/3290581986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f001987a0 0x7f0f0019cae0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.595+0000 7f0f04ba8700 1 -- 172.21.15.190:0/3290581986 >> 172.21.15.190:0/3290581986 conn(0x7f0f000fcf80 msgr2=0x7f0f001064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.595+0000 7f0f04ba8700 1 -- 172.21.15.190:0/3290581986 shutdown_connections 2024-04-21T10:22:53.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:53.595+0000 7f0f04ba8700 1 -- 172.21.15.190:0/3290581986 wait complete. 2024-04-21T10:22:53.596 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2024-04-21T10:22:54.060 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 podman[31449]: 2024-04-21 10:22:54.037129835 +0000 UTC m=+0.342669700 container start 4f5a02b35bd3c649960d8ea0e14a814e9571913c4a84f721350dac8cf0d551b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-21T10:22:54.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: set uid:gid to 167:167 (ceph:ceph) 2024-04-21T10:22:54.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev), process ceph-mon, pid 2 2024-04-21T10:22:54.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pidfile_write: ignore empty --pid-file 2024-04-21T10:22:54.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: load: jerasure load: lrc 2024-04-21T10:22:54.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: RocksDB version: 7.9.2 2024-04-21T10:22:54.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Git sha 0 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Compile date 2024-04-19 14:19:22 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: DB SUMMARY 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: DB Session ID: CCNN5KY0UJXRHM3OAMGV 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: CURRENT file: CURRENT 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: IDENTITY file: IDENTITY 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.error_if_exists: 0 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.create_if_missing: 0 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.paranoid_checks: 1 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-21T10:22:54.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.env: 0x5593746e2cc0 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.fs: PosixFileSystem 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.info_log: 0x559375e5fbe0 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_file_opening_threads: 16 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.statistics: (nil) 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.use_fsync: 0 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_log_file_size: 0 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.keep_log_file_num: 1000 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.recycle_log_file_num: 0 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.allow_fallocate: 1 2024-04-21T10:22:54.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.allow_mmap_reads: 0 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.allow_mmap_writes: 0 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.use_direct_reads: 0 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.create_missing_column_families: 0 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.db_log_dir: 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.wal_dir: 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.advise_random_on_open: 1 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.db_write_buffer_size: 0 2024-04-21T10:22:54.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.write_buffer_manager: 0x559375df30e0 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.rate_limiter: (nil) 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.wal_recovery_mode: 2 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.enable_thread_tracking: 0 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.enable_pipelined_write: 0 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.unordered_write: 0 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.row_cache: None 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.wal_filter: None 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-21T10:22:54.453 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.allow_ingest_behind: 0 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.two_write_queues: 0 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.manual_wal_flush: 0 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.wal_compression: 0 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.atomic_flush: 0 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.log_readahead_size: 0 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.best_efforts_recovery: 0 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.allow_data_in_errors: 0 2024-04-21T10:22:54.454 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.db_host_id: __hostname__ 2024-04-21T10:22:54.455 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-21T10:22:54.456 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_background_jobs: 2 2024-04-21T10:22:54.456 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_background_compactions: -1 2024-04-21T10:22:54.456 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_subcompactions: 1 2024-04-21T10:22:54.456 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-21T10:22:54.456 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_total_wal_size: 0 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_open_files: -1 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bytes_per_sync: 0 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_readahead_size: 0 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_background_flushes: -1 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Compression algorithms supported: 2024-04-21T10:22:54.457 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: kZSTD supported: 0 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: kXpressCompression supported: 0 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: kLZ4HCCompression supported: 1 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: kZlibCompression supported: 1 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: kSnappyCompression supported: 1 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: kLZ4Compression supported: 1 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: kBZip2Compression supported: 0 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.merge_operator: 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_filter: None 2024-04-21T10:22:54.458 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_filter_factory: None 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.sst_partitioner_factory: None 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559375e5fdc0) 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout: cache_index_and_filter_blocks: 1 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout: pin_top_level_index_and_filter: 1 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout: index_type: 0 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout: data_block_index_type: 0 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout: index_shortening: 1 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout: checksum: 4 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout: no_block_cache: 0 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_cache: 0x559375e251f0 2024-04-21T10:22:54.459 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_cache_name: BinnedLRUCache 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_cache_options: 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: capacity : 536870912 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: num_shard_bits : 4 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: strict_capacity_limit : 0 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: high_pri_pool_ratio: 0.000 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_cache_compressed: (nil) 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: persistent_cache: (nil) 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_size: 4096 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_size_deviation: 10 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_restart_interval: 16 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: index_block_restart_interval: 1 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: metadata_block_size: 4096 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: partition_filters: 0 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: use_delta_encoding: 1 2024-04-21T10:22:54.460 INFO:journalctl@ceph.mon.b.smithi169.stdout: filter_policy: bloomfilter 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout: whole_key_filtering: 1 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout: verify_compression: 0 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout: read_amp_bytes_per_bit: 0 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout: format_version: 5 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout: enable_index_compression: 1 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout: block_align: 0 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout: max_auto_readahead_size: 262144 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout: prepopulate_block_cache: 0 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout: initial_auto_readahead_size: 8192 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout: num_file_reads_for_auto_readahead: 2 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.write_buffer_size: 33554432 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_write_buffer_number: 2 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compression: NoCompression 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bottommost_compression: Disabled 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.prefix_extractor: nullptr 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-21T10:22:54.461 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.num_levels: 7 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compression_opts.level: 32767 2024-04-21T10:22:54.462 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compression_opts.strategy: 0 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compression_opts.enabled: false 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.target_file_size_base: 67108864 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-21T10:22:54.463 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.arena_block_size: 1048576 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.disable_auto_compactions: 0 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-21T10:22:54.464 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.inplace_update_support: 0 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-21T10:22:54.465 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.bloom_locality: 0 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.max_successive_merges: 0 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.paranoid_file_checks: 0 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.force_consistency_checks: 1 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.report_bg_io_stats: 0 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.ttl: 2592000 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.enable_blob_files: false 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.min_blob_size: 0 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.blob_file_size: 268435456 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-21T10:22:54.466 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.blob_file_starting_level: 0 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f8fa857b-4d75-4ff1-9477-eb62f244e530 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713694974066922, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713694974067352, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713694974, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f8fa857b-4d75-4ff1-9477-eb62f244e530", "db_session_id": "CCNN5KY0UJXRHM3OAMGV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713694974067431, "job": 1, "event": "recovery_finished"} 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559375f3a000 2024-04-21T10:22:54.467 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: DB pointer 0x559375f24000 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout: ** DB Stats ** 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout: ** Compaction Stats [default] ** 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T10:22:54.468 INFO:journalctl@ceph.mon.b.smithi169.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: ** Compaction Stats [default] ** 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-21T10:22:54.469 INFO:journalctl@ceph.mon.b.smithi169.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout: Block cache BinnedLRUCache@0x559375e251f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: using public_addr v2:172.21.15.169:0/0 -> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: starting mon.b rank -1 at public addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] at bind addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(???) e0 preinit fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:22:54.470 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).mds e1 new map 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout: e1 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout: legacy client fscid: -1 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout: 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout: No filesystems configured 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-21T10:22:54.471 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Deploying cephadm binary to smithi190 2024-04-21T10:22:54.472 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Added host smithi190 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.473 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/45354162' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/45354162' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Updating smithi190:/etc/ceph/ceph.conf 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:22:54.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi169:172.21.15.169=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Saving service mon spec with placement smithi026:172.21.15.26=a;smithi169:172.21.15.169=b;smithi190:172.21.15.190=c;count:3 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:22:54.475 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Deploying daemon mon.c on smithi190 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.a calling monitor election 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='client.? 172.21.15.190:0/509684' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.c calling monitor election 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:54.476 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: monmap epoch 2 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: last_changed 2024-04-21T10:22:44.967557+0000 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: created 2024-04-21T10:21:05.761117+0000 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: min_mon_release 19 (squid) 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: election_strategy: 1 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: fsmap 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mgrmap e13: x(active, since 47s) 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: overall HEALTH_OK 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.477 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:54.478 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:22:54.478 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:22:54.478 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: Deploying daemon mon.b on smithi169 2024-04-21T10:22:54.478 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:54.478 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:54.478 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:54.478 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 ceph-mon[31469]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-21T10:22:54.478 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 bash[31449]: 4f5a02b35bd3c649960d8ea0e14a814e9571913c4a84f721350dac8cf0d551b3 2024-04-21T10:22:54.478 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:22:54 smithi169 systemd[1]: Started Ceph mon.b for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:22:55.202 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T10:22:55.203 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T10:22:55.439 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:22:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:22:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: mon.a calling monitor election 2024-04-21T10:22:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: mon.c calling monitor election 2024-04-21T10:22:59.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:22:59.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:59.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:59.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:22:59.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:22:59.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:59.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:22:59.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:22:59.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:59.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:22:59.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: monmap epoch 3 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: last_changed 2024-04-21T10:22:54.089611+0000 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: created 2024-04-21T10:21:05.761117+0000 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: min_mon_release 19 (squid) 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: election_strategy: 1 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: 2: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: fsmap 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: mgrmap e13: x(active, since 57s) 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: overall HEALTH_OK 2024-04-21T10:22:59.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:59.438 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:59.438 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:59.438 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:22:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:59.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:22:59.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: mon.a calling monitor election 2024-04-21T10:22:59.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: mon.c calling monitor election 2024-04-21T10:22:59.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:22:59.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:22:59.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: monmap epoch 3 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: last_changed 2024-04-21T10:22:54.089611+0000 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: created 2024-04-21T10:21:05.761117+0000 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: min_mon_release 19 (squid) 2024-04-21T10:22:59.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: election_strategy: 1 2024-04-21T10:22:59.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-04-21T10:22:59.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-04-21T10:22:59.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: 2: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-04-21T10:22:59.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: fsmap 2024-04-21T10:22:59.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T10:22:59.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: mgrmap e13: x(active, since 57s) 2024-04-21T10:22:59.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: overall HEALTH_OK 2024-04-21T10:22:59.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:59.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:59.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:59.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:22:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:22:59.829 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.829+0000 7fd002eac700 1 -- 172.21.15.190:0/1914743773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfec0035c0 msgr2=0x7fcfec005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:22:59.829 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.829+0000 7fd002eac700 1 --2- 172.21.15.190:0/1914743773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfec0035c0 0x7fcfec005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcffc06acb0 tx=0x7fcff000feb0 comp rx=0 tx=0).stop 2024-04-21T10:22:59.829 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.829+0000 7fd002eac700 1 -- 172.21.15.190:0/1914743773 shutdown_connections 2024-04-21T10:22:59.829 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.829+0000 7fd002eac700 1 --2- 172.21.15.190:0/1914743773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfec0035c0 0x7fcfec005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:59.829 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.829+0000 7fd002eac700 1 -- 172.21.15.190:0/1914743773 >> 172.21.15.190:0/1914743773 conn(0x7fcffc0fcf40 msgr2=0x7fcffc0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:22:59.829 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.829+0000 7fd002eac700 1 -- 172.21.15.190:0/1914743773 shutdown_connections 2024-04-21T10:22:59.829 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.829+0000 7fd002eac700 1 -- 172.21.15.190:0/1914743773 wait complete. 2024-04-21T10:22:59.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.830+0000 7fd002eac700 1 Processor -- start 2024-04-21T10:22:59.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.830+0000 7fd002eac700 1 -- start start 2024-04-21T10:22:59.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.830+0000 7fd002eac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfec0035c0 0x7fcffc198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:59.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.830+0000 7fd002eac700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc1986a0 0x7fcffc19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:59.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.830+0000 7fd002eac700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcffc19cf70 0x7fcffc19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:59.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.830+0000 7fd002eac700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10a4a0 con 0x7fcfec0035c0 2024-04-21T10:22:59.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.830+0000 7fd002eac700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10a320 con 0x7fcffc19cf70 2024-04-21T10:22:59.832 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.830+0000 7fd002eac700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc10a620 con 0x7fcffc1986a0 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.830+0000 7fcffbfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc1986a0 0x7fcffc19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fcffbfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc1986a0 0x7fcffc19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:41976/0 (socket says 172.21.15.190:41976) 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fcffbfff700 1 -- 172.21.15.190:0/3785999166 learned_addr learned my addr 172.21.15.190:0/3785999166 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd001449700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcffc19cf70 0x7fcffc19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd000c48700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfec0035c0 0x7fcffc198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fcffbfff700 1 -- 172.21.15.190:0/3785999166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc1986a0 msgr2=0x7fcffc19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fcffbfff700 1 -- 172.21.15.190:0/3785999166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc1986a0 msgr2=0x7fcffc19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fcffbfff700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc1986a0 0x7fcffc19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fcffbfff700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc1986a0 0x7fcffc19ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd001449700 1 -- 172.21.15.190:0/3785999166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcffc19cf70 msgr2=0x7fcffc19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd001449700 1 -- 172.21.15.190:0/3785999166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcffc19cf70 msgr2=0x7fcffc19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd001449700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcffc19cf70 0x7fcffc19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd001449700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcffc19cf70 0x7fcffc19d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T10:22:59.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd000c48700 1 -- 172.21.15.190:0/3785999166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc1986a0 msgr2=0x7fcffc19ca30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T10:22:59.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd000c48700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc1986a0 0x7fcffc19ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:59.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd000c48700 1 -- 172.21.15.190:0/3785999166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcffc19cf70 msgr2=0x7fcffc19d420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T10:22:59.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd000c48700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcffc19cf70 0x7fcffc19d420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:22:59.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd000c48700 1 -- 172.21.15.190:0/3785999166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc1a4ce0 con 0x7fcfec0035c0 2024-04-21T10:22:59.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd000c48700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfec0035c0 0x7fcffc198160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcff0010010 tx=0x7fcff0013f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:59.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fcff9ffb700 1 -- 172.21.15.190:0/3785999166 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0028070 con 0x7fcfec0035c0 2024-04-21T10:22:59.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fcff9ffb700 1 -- 172.21.15.190:0/3785999166 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff0005020 con 0x7fcfec0035c0 2024-04-21T10:22:59.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fcff9ffb700 1 -- 172.21.15.190:0/3785999166 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0011e60 con 0x7fcfec0035c0 2024-04-21T10:22:59.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.831+0000 7fd002eac700 1 -- 172.21.15.190:0/3785999166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcffc1a4eb0 con 0x7fcfec0035c0 2024-04-21T10:22:59.834 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.832+0000 7fd002eac700 1 -- 172.21.15.190:0/3785999166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcffc1a5310 con 0x7fcfec0035c0 2024-04-21T10:22:59.835 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.833+0000 7fd002eac700 1 -- 172.21.15.190:0/3785999166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcffc0640a0 con 0x7fcfec0035c0 2024-04-21T10:22:59.835 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.833+0000 7fcff9ffb700 1 -- 172.21.15.190:0/3785999166 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcff002b070 con 0x7fcfec0035c0 2024-04-21T10:22:59.835 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.833+0000 7fcff9ffb700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcfe0039ae0 0x7fcfe003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:22:59.835 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.833+0000 7fcff9ffb700 1 -- 172.21.15.190:0/3785999166 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcff004e860 con 0x7fcfec0035c0 2024-04-21T10:22:59.835 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.833+0000 7fcffbfff700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcfe0039ae0 0x7fcfe003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:22:59.835 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.834+0000 7fcffbfff700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcfe0039ae0 0x7fcfe003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009180 tx=0x7fcfe8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:22:59.838 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:22:59.839+0000 7fcff9ffb700 1 -- 172.21.15.190:0/3785999166 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcff001a390 con 0x7fcfec0035c0 2024-04-21T10:23:00.138 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.138+0000 7fd002eac700 1 -- 172.21.15.190:0/3785999166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcffc02e140 con 0x7fcfec0035c0 2024-04-21T10:23:00.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.139+0000 7fcff9ffb700 1 -- 172.21.15.190:0/3785999166 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fcff0034db0 con 0x7fcfec0035c0 2024-04-21T10:23:00.139 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-21T10:23:00.139 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":3,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","modified":"2024-04-21T10:22:54.089611Z","created":"2024-04-21T10:21:05.761117Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-21T10:23:00.141 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.141+0000 7fd002eac700 1 -- 172.21.15.190:0/3785999166 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcfe0039ae0 msgr2=0x7fcfe003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:00.141 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.141+0000 7fd002eac700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcfe0039ae0 0x7fcfe003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009180 tx=0x7fcfe8008040 comp rx=0 tx=0).stop 2024-04-21T10:23:00.141 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.141+0000 7fd002eac700 1 -- 172.21.15.190:0/3785999166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfec0035c0 msgr2=0x7fcffc198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:00.141 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.141+0000 7fd002eac700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfec0035c0 0x7fcffc198160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcff0010010 tx=0x7fcff0013f60 comp rx=0 tx=0).stop 2024-04-21T10:23:00.141 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.141+0000 7fd002eac700 1 -- 172.21.15.190:0/3785999166 shutdown_connections 2024-04-21T10:23:00.141 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.141+0000 7fd002eac700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcfe0039ae0 0x7fcfe003bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:00.141 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.141+0000 7fd002eac700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcffc19cf70 0x7fcffc19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:00.141 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.141+0000 7fd002eac700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfec0035c0 0x7fcffc198160 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:00.142 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.141+0000 7fd002eac700 1 --2- 172.21.15.190:0/3785999166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcffc1986a0 0x7fcffc19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:00.142 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.141+0000 7fd002eac700 1 -- 172.21.15.190:0/3785999166 >> 172.21.15.190:0/3785999166 conn(0x7fcffc0fcf40 msgr2=0x7fcffc107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:23:00.142 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.141+0000 7fd002eac700 1 -- 172.21.15.190:0/3785999166 shutdown_connections 2024-04-21T10:23:00.142 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:00.141+0000 7fd002eac700 1 -- 172.21.15.190:0/3785999166 wait complete. 2024-04-21T10:23:00.142 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 3 2024-04-21T10:23:00.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:00.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:23:00.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:00 smithi026 ceph-mon[27965]: from='client.? 172.21.15.190:0/3785999166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T10:23:00.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:00.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:23:00.499 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:00 smithi190 ceph-mon[31423]: from='client.? 172.21.15.190:0/3785999166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T10:23:00.784 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-21T10:23:00.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-21T10:23:01.040 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:23:01.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: mon.a calling monitor election 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: mon.c calling monitor election 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:23:01.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: monmap epoch 3 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: last_changed 2024-04-21T10:22:54.089611+0000 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: created 2024-04-21T10:21:05.761117+0000 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: min_mon_release 19 (squid) 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: election_strategy: 1 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: 2: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: fsmap 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: mgrmap e13: x(active, since 57s) 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: overall HEALTH_OK 2024-04-21T10:23:01.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:01.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:01.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:01.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:01.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:01.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:23:01.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='client.? 172.21.15.190:0/3785999166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T10:23:02.271 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: mon.b calling monitor election 2024-04-21T10:23:02.271 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: mon.b calling monitor election 2024-04-21T10:23:02.271 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: mon.c calling monitor election 2024-04-21T10:23:02.271 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: mon.a calling monitor election 2024-04-21T10:23:02.271 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T10:23:02.271 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: monmap epoch 3 2024-04-21T10:23:02.272 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:23:02.272 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: last_changed 2024-04-21T10:22:54.089611+0000 2024-04-21T10:23:02.272 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: created 2024-04-21T10:21:05.761117+0000 2024-04-21T10:23:02.272 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: min_mon_release 19 (squid) 2024-04-21T10:23:02.272 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: election_strategy: 1 2024-04-21T10:23:02.272 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-04-21T10:23:02.272 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-04-21T10:23:02.273 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: 2: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-04-21T10:23:02.273 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: fsmap 2024-04-21T10:23:02.273 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T10:23:02.273 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: mgrmap e13: x(active, since 59s) 2024-04-21T10:23:02.273 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: overall HEALTH_OK 2024-04-21T10:23:02.273 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.273 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.273 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:02.274 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:02.274 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: Updating smithi026:/etc/ceph/ceph.conf 2024-04-21T10:23:02.274 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-21T10:23:02.274 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: Updating smithi190:/etc/ceph/ceph.conf 2024-04-21T10:23:02.274 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:23:02.274 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: Updating smithi026:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:23:02.275 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:23:02.275 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.275 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.275 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.275 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.275 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.276 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.276 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.276 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.276 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.276 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.276 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.276 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T10:23:02.277 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:23:02.277 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T10:23:02.277 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:02.277 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:01 smithi026 ceph-mon[27965]: Reconfiguring daemon mon.a on smithi026 2024-04-21T10:23:02.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: mon.b calling monitor election 2024-04-21T10:23:02.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: mon.b calling monitor election 2024-04-21T10:23:02.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: mon.c calling monitor election 2024-04-21T10:23:02.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: mon.a calling monitor election 2024-04-21T10:23:02.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T10:23:02.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: monmap epoch 3 2024-04-21T10:23:02.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:23:02.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: last_changed 2024-04-21T10:22:54.089611+0000 2024-04-21T10:23:02.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: created 2024-04-21T10:21:05.761117+0000 2024-04-21T10:23:02.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: min_mon_release 19 (squid) 2024-04-21T10:23:02.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: election_strategy: 1 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: 2: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: fsmap 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: mgrmap e13: x(active, since 59s) 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: overall HEALTH_OK 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: Updating smithi026:/etc/ceph/ceph.conf 2024-04-21T10:23:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: Updating smithi190:/etc/ceph/ceph.conf 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: Updating smithi026:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T10:23:02.280 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:23:02.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T10:23:02.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:02.281 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:01 smithi190 ceph-mon[31423]: Reconfiguring daemon mon.a on smithi026 2024-04-21T10:23:02.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: mon.b calling monitor election 2024-04-21T10:23:02.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: mon.b calling monitor election 2024-04-21T10:23:02.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: mon.c calling monitor election 2024-04-21T10:23:02.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: mon.a calling monitor election 2024-04-21T10:23:02.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T10:23:02.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: monmap epoch 3 2024-04-21T10:23:02.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:23:02.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: last_changed 2024-04-21T10:22:54.089611+0000 2024-04-21T10:23:02.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: created 2024-04-21T10:21:05.761117+0000 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: min_mon_release 19 (squid) 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: election_strategy: 1 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: 0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.c 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: 2: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: fsmap 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: mgrmap e13: x(active, since 59s) 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: overall HEALTH_OK 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:02.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: Updating smithi026:/etc/ceph/ceph.conf 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: Updating smithi190:/etc/ceph/ceph.conf 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: Updating smithi026:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.451 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:02.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T10:23:02.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:23:02.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T10:23:02.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:02.452 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:01 smithi169 ceph-mon[31469]: Reconfiguring daemon mon.a on smithi026 2024-04-21T10:23:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.586+0000 7f6f240d7700 1 -- 172.21.15.26:0/690323802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f1c108550 msgr2=0x7f6f1c108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.586+0000 7f6f240d7700 1 --2- 172.21.15.26:0/690323802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f1c108550 0x7f6f1c108930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009910 tx=0x7f6f18009c20 comp rx=0 tx=0).stop 2024-04-21T10:23:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.586+0000 7f6f240d7700 1 -- 172.21.15.26:0/690323802 shutdown_connections 2024-04-21T10:23:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.586+0000 7f6f240d7700 1 --2- 172.21.15.26:0/690323802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f1c108550 0x7f6f1c108930 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.586+0000 7f6f240d7700 1 -- 172.21.15.26:0/690323802 >> 172.21.15.26:0/690323802 conn(0x7f6f1c0fdf70 msgr2=0x7f6f1c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:23:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.587+0000 7f6f240d7700 1 -- 172.21.15.26:0/690323802 shutdown_connections 2024-04-21T10:23:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.587+0000 7f6f240d7700 1 -- 172.21.15.26:0/690323802 wait complete. 2024-04-21T10:23:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.587+0000 7f6f240d7700 1 Processor -- start 2024-04-21T10:23:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.587+0000 7f6f240d7700 1 -- start start 2024-04-21T10:23:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.587+0000 7f6f240d7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f1c108550 0x7f6f1c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f240d7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f1c075280 0x7f6f1c075700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f240d7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1c1a4250 0x7f6f1c1a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f240d7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10c9c0 con 0x7f6f1c108550 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f240d7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10c840 con 0x7f6f1c075280 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f240d7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10cb40 con 0x7f6f1c1a4250 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f21e73700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f1c108550 0x7f6f1c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f21e73700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f1c108550 0x7f6f1c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47940/0 (socket says 172.21.15.26:47940) 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f21e73700 1 -- 172.21.15.26:0/2685719970 learned_addr learned my addr 172.21.15.26:0/2685719970 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f21672700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f1c075280 0x7f6f1c075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f22674700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1c1a4250 0x7f6f1c1a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f21e73700 1 -- 172.21.15.26:0/2685719970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1c1a4250 msgr2=0x7f6f1c1a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f21e73700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1c1a4250 0x7f6f1c1a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f21e73700 1 -- 172.21.15.26:0/2685719970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f1c075280 msgr2=0x7f6f1c075700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f21e73700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f1c075280 0x7f6f1c075700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f21e73700 1 -- 172.21.15.26:0/2685719970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c1a6d70 con 0x7f6f1c108550 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f21672700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f1c075280 0x7f6f1c075700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:23:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f22674700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1c1a4250 0x7f6f1c1a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:23:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f21e73700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f1c108550 0x7f6f1c074d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6f18005b40 tx=0x7f6f18005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:23:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f0effd700 1 -- 172.21.15.26:0/2685719970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18026070 con 0x7f6f1c108550 2024-04-21T10:23:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f0effd700 1 -- 172.21.15.26:0/2685719970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f1800b750 con 0x7f6f1c108550 2024-04-21T10:23:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f0effd700 1 -- 172.21.15.26:0/2685719970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f180038e0 con 0x7f6f1c108550 2024-04-21T10:23:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.588+0000 7f6f240d7700 1 -- 172.21.15.26:0/2685719970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c1a7000 con 0x7f6f1c108550 2024-04-21T10:23:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.589+0000 7f6f240d7700 1 -- 172.21.15.26:0/2685719970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c1a7490 con 0x7f6f1c108550 2024-04-21T10:23:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.589+0000 7f6f0effd700 1 -- 172.21.15.26:0/2685719970 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6f18029070 con 0x7f6f1c108550 2024-04-21T10:23:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.589+0000 7f6f240d7700 1 -- 172.21.15.26:0/2685719970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f1c0640a0 con 0x7f6f1c108550 2024-04-21T10:23:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.589+0000 7f6f0effd700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6f08039ae0 0x7f6f0803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.590+0000 7f6f0effd700 1 -- 172.21.15.26:0/2685719970 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6f1804d520 con 0x7f6f1c108550 2024-04-21T10:23:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.590+0000 7f6f21672700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6f08039ae0 0x7f6f0803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.590+0000 7f6f21672700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6f08039ae0 0x7f6f0803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009910 tx=0x7f6f10008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:23:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.595+0000 7f6f0effd700 1 -- 172.21.15.26:0/2685719970 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f18013070 con 0x7f6f1c108550 2024-04-21T10:23:02.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.803+0000 7f6f240d7700 1 -- 172.21.15.26:0/2685719970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6f1c1a7900 con 0x7f6f1c108550 2024-04-21T10:23:02.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.803+0000 7f6f0effd700 1 -- 172.21.15.26:0/2685719970 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f6f180043d0 con 0x7f6f1c108550 2024-04-21T10:23:02.803 INFO:teuthology.orchestra.run.smithi026.stdout:# minimal ceph.conf for b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:23:02.803 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2024-04-21T10:23:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:23:02.803 INFO:teuthology.orchestra.run.smithi026.stdout: mon_host = [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2024-04-21T10:23:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.805+0000 7f6f240d7700 1 -- 172.21.15.26:0/2685719970 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6f08039ae0 msgr2=0x7f6f0803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.805+0000 7f6f240d7700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6f08039ae0 0x7f6f0803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009910 tx=0x7f6f10008040 comp rx=0 tx=0).stop 2024-04-21T10:23:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.805+0000 7f6f240d7700 1 -- 172.21.15.26:0/2685719970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f1c108550 msgr2=0x7f6f1c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:02.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.805+0000 7f6f240d7700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f1c108550 0x7f6f1c074d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6f18005b40 tx=0x7f6f18005030 comp rx=0 tx=0).stop 2024-04-21T10:23:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.805+0000 7f6f240d7700 1 -- 172.21.15.26:0/2685719970 shutdown_connections 2024-04-21T10:23:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.805+0000 7f6f240d7700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6f08039ae0 0x7f6f0803bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.805+0000 7f6f240d7700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f1c1a4250 0x7f6f1c1a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.805+0000 7f6f240d7700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f1c108550 0x7f6f1c074d40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.805+0000 7f6f240d7700 1 --2- 172.21.15.26:0/2685719970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f1c075280 0x7f6f1c075700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.805+0000 7f6f240d7700 1 -- 172.21.15.26:0/2685719970 >> 172.21.15.26:0/2685719970 conn(0x7f6f1c0fdf70 msgr2=0x7f6f1c0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:23:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.805+0000 7f6f240d7700 1 -- 172.21.15.26:0/2685719970 shutdown_connections 2024-04-21T10:23:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:02.805+0000 7f6f240d7700 1 -- 172.21.15.26:0/2685719970 wait complete. 2024-04-21T10:23:03.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:23:03.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:02 smithi026 ceph-mon[27965]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:03.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:03.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:02 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2685719970' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:03.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:23:03.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:02 smithi190 ceph-mon[31423]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:03.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:03.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:02 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2685719970' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:03.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:23:03.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:02 smithi169 ceph-mon[31469]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:03.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:03.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:02 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2685719970' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:04.179 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-21T10:23:04.179 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:23:04.179 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T10:23:04.207 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:23:04.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:23:04.273 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-21T10:23:04.273 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T10:23:04.298 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-21T10:23:04.299 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:23:04.362 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-21T10:23:04.363 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T10:23:04.393 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-21T10:23:04.393 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:23:04.461 INFO:tasks.cephadm:Adding mgr.x on smithi026 2024-04-21T10:23:04.461 INFO:tasks.cephadm:Adding mgr.y on smithi169 2024-04-21T10:23:04.461 INFO:tasks.cephadm:Adding mgr.z on smithi190 2024-04-21T10:23:04.461 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi026=x;smithi169=y;smithi190=z' 2024-04-21T10:23:04.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:04 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:04.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:04 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:04.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:04 smithi169 ceph-mon[31469]: Reconfiguring mon.b (monmap changed)... 2024-04-21T10:23:04.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:04 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:23:04.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:04 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T10:23:04.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:04 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:04.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:04 smithi169 ceph-mon[31469]: Reconfiguring daemon mon.b on smithi169 2024-04-21T10:23:04.738 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:23:04.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:04 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:04.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:04 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:04.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:04 smithi190 ceph-mon[31423]: Reconfiguring mon.b (monmap changed)... 2024-04-21T10:23:04.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:04 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:23:04.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:04 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T10:23:04.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:04 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:04.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:04 smithi190 ceph-mon[31423]: Reconfiguring daemon mon.b on smithi169 2024-04-21T10:23:04.828 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:04 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:04.828 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:04 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:04.828 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:04 smithi026 ceph-mon[27965]: Reconfiguring mon.b (monmap changed)... 2024-04-21T10:23:04.829 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:04 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:23:04.829 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:04 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T10:23:04.829 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:04 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:04.829 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:04 smithi026 ceph-mon[27965]: Reconfiguring daemon mon.b on smithi169 2024-04-21T10:23:05.897 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:05 smithi190 ceph-mon[31423]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:05.897 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:05 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:05.897 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:05 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:05.897 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:05 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:23:05.897 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:05 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T10:23:05.897 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:05 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:05.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:05 smithi026 ceph-mon[27965]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:05.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:05.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:05.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:23:05.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T10:23:05.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:05 smithi169 ceph-mon[31469]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:05 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:05 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:05 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T10:23:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:05 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T10:23:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:05 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:06.297 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.295+0000 7f21a74c3700 1 -- 172.21.15.190:0/1088014831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a01030e0 msgr2=0x7f21a01054d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:06.297 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.295+0000 7f21a74c3700 1 --2- 172.21.15.190:0/1088014831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a01030e0 0x7f21a01054d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f219c00b0a0 tx=0x7f219c00b3b0 comp rx=0 tx=0).stop 2024-04-21T10:23:06.297 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.295+0000 7f21a74c3700 1 -- 172.21.15.190:0/1088014831 shutdown_connections 2024-04-21T10:23:06.297 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.295+0000 7f21a74c3700 1 --2- 172.21.15.190:0/1088014831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a01030e0 0x7f21a01054d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:06.297 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.295+0000 7f21a74c3700 1 -- 172.21.15.190:0/1088014831 >> 172.21.15.190:0/1088014831 conn(0x7f21a00fca50 msgr2=0x7f21a00fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.295+0000 7f21a74c3700 1 -- 172.21.15.190:0/1088014831 shutdown_connections 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.295+0000 7f21a74c3700 1 -- 172.21.15.190:0/1088014831 wait complete. 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a74c3700 1 Processor -- start 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a74c3700 1 -- start start 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a74c3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a01030e0 0x7f21a0197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a74c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a0198240 0x7f21a019d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a74c3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21a01986c0 0x7f21a019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a74c3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21a01084a0 con 0x7f21a0198240 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a74c3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0108320 con 0x7f21a01030e0 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a74c3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0108620 con 0x7f21a01986c0 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a5a60700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21a01986c0 0x7f21a019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a5a60700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21a01986c0 0x7f21a019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:41986/0 (socket says 172.21.15.190:41986) 2024-04-21T10:23:06.298 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a5a60700 1 -- 172.21.15.190:0/2379112183 learned_addr learned my addr 172.21.15.190:0/2379112183 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a4a5e700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a0198240 0x7f21a019d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.296+0000 7f21a525f700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a01030e0 0x7f21a0197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.297+0000 7f21a5a60700 1 -- 172.21.15.190:0/2379112183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a01030e0 msgr2=0x7f21a0197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.297+0000 7f21a5a60700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a01030e0 0x7f21a0197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.297+0000 7f21a5a60700 1 -- 172.21.15.190:0/2379112183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a0198240 msgr2=0x7f21a019d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.297+0000 7f21a5a60700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a0198240 0x7f21a019d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.297+0000 7f21a5a60700 1 -- 172.21.15.190:0/2379112183 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a01a0120 con 0x7f21a01986c0 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.297+0000 7f21a4a5e700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a0198240 0x7f21a019d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.297+0000 7f21a5a60700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21a01986c0 0x7f21a019f820 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f219800b550 tx=0x7f219800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.297+0000 7f21967fc700 1 -- 172.21.15.190:0/2379112183 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198015020 con 0x7f21a01986c0 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.297+0000 7f21a74c3700 1 -- 172.21.15.190:0/2379112183 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a01a0410 con 0x7f21a01986c0 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.297+0000 7f21967fc700 1 -- 172.21.15.190:0/2379112183 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2198011ce0 con 0x7f21a01986c0 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.297+0000 7f21967fc700 1 -- 172.21.15.190:0/2379112183 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198010da0 con 0x7f21a01986c0 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.297+0000 7f21a74c3700 1 -- 172.21.15.190:0/2379112183 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a01a0980 con 0x7f21a01986c0 2024-04-21T10:23:06.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.298+0000 7f21967fc700 1 -- 172.21.15.190:0/2379112183 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2198018020 con 0x7f21a01986c0 2024-04-21T10:23:06.300 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.298+0000 7f21a74c3700 1 -- 172.21.15.190:0/2379112183 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a00640a0 con 0x7f21a01986c0 2024-04-21T10:23:06.300 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.298+0000 7f21967fc700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f218c039ae0 0x7f218c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:06.300 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.298+0000 7f21967fc700 1 -- 172.21.15.190:0/2379112183 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f219804c270 con 0x7f21a01986c0 2024-04-21T10:23:06.300 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.298+0000 7f21a525f700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f218c039ae0 0x7f218c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:06.300 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.299+0000 7f21a525f700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f218c039ae0 0x7f218c03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f219c0062a0 tx=0x7f219c006230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:23:06.303 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.303+0000 7f21967fc700 1 -- 172.21.15.190:0/2379112183 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f219801f020 con 0x7f21a01986c0 2024-04-21T10:23:06.517 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.518+0000 7f21a74c3700 1 -- 172.21.15.190:0/2379112183 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi169=y;smithi190=z", "target": ["mon-mgr", ""]}) -- 0x7f21a0062f10 con 0x7f218c039ae0 2024-04-21T10:23:06.524 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.524+0000 7f21967fc700 1 -- 172.21.15.190:0/2379112183 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f21a0062f10 con 0x7f218c039ae0 2024-04-21T10:23:06.524 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mgr update... 2024-04-21T10:23:06.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.526+0000 7f21a74c3700 1 -- 172.21.15.190:0/2379112183 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f218c039ae0 msgr2=0x7f218c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:06.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.526+0000 7f21a74c3700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f218c039ae0 0x7f218c03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f219c0062a0 tx=0x7f219c006230 comp rx=0 tx=0).stop 2024-04-21T10:23:06.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.526+0000 7f21a74c3700 1 -- 172.21.15.190:0/2379112183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21a01986c0 msgr2=0x7f21a019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:06.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.526+0000 7f21a74c3700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21a01986c0 0x7f21a019f820 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f219800b550 tx=0x7f219800b910 comp rx=0 tx=0).stop 2024-04-21T10:23:06.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.526+0000 7f21a74c3700 1 -- 172.21.15.190:0/2379112183 shutdown_connections 2024-04-21T10:23:06.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.526+0000 7f21a74c3700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f218c039ae0 0x7f218c03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:06.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.526+0000 7f21a74c3700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21a01030e0 0x7f21a0197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:06.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.526+0000 7f21a74c3700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21a01986c0 0x7f21a019f820 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:06.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.526+0000 7f21a74c3700 1 --2- 172.21.15.190:0/2379112183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a0198240 0x7f21a019d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:06.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.526+0000 7f21a74c3700 1 -- 172.21.15.190:0/2379112183 >> 172.21.15.190:0/2379112183 conn(0x7f21a00fca50 msgr2=0x7f21a00fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:23:06.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.527+0000 7f21a74c3700 1 -- 172.21.15.190:0/2379112183 shutdown_connections 2024-04-21T10:23:06.527 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:23:06.527+0000 7f21a74c3700 1 -- 172.21.15.190:0/2379112183 wait complete. 2024-04-21T10:23:06.739 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:06 smithi190 ceph-mon[31423]: Reconfiguring mon.c (monmap changed)... 2024-04-21T10:23:06.739 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:06 smithi190 ceph-mon[31423]: Reconfiguring daemon mon.c on smithi190 2024-04-21T10:23:06.740 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:06.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:06 smithi026 ceph-mon[27965]: Reconfiguring mon.c (monmap changed)... 2024-04-21T10:23:06.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:06 smithi026 ceph-mon[27965]: Reconfiguring daemon mon.c on smithi190 2024-04-21T10:23:06.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:06.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:06 smithi169 ceph-mon[31469]: Reconfiguring mon.c (monmap changed)... 2024-04-21T10:23:06.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:06 smithi169 ceph-mon[31469]: Reconfiguring daemon mon.c on smithi190 2024-04-21T10:23:06.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:07.586 DEBUG:teuthology.orchestra.run.smithi169:mgr.y> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.y.service 2024-04-21T10:23:07.589 DEBUG:teuthology.orchestra.run.smithi190:mgr.z> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.z.service 2024-04-21T10:23:07.591 INFO:tasks.cephadm:Deploying OSDs... 2024-04-21T10:23:07.591 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:23:07.591 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T10:23:07.611 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T10:23:07.611 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-04-21T10:23:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T10:23:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T10:23:07.669 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-21T10:23:07.669 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:23:07.669 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T10:23:07.669 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-21 10:22:12.021071226 +0000 2024-04-21T10:23:07.669 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-21 10:17:22.080698803 +0000 2024-04-21T10:23:07.670 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-21 10:17:22.080698803 +0000 2024-04-21T10:23:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-21 10:17:22.080698803 +0000 2024-04-21T10:23:07.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T10:23:07.736 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-04-21T10:23:07.737 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-04-21T10:23:07.737 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000122675 s, 4.2 MB/s 2024-04-21T10:23:07.738 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T10:23:07.797 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-04-21T10:23:07.855 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T10:23:07.855 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T10:23:07.855 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-21T10:23:07.855 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:23:07.855 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T10:23:07.855 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-21 10:22:12.050070795 +0000 2024-04-21T10:23:07.855 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-21 10:17:22.418693378 +0000 2024-04-21T10:23:07.856 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-21 10:17:22.418693378 +0000 2024-04-21T10:23:07.856 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-21 10:17:22.418693378 +0000 2024-04-21T10:23:07.856 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T10:23:07.897 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:07 smithi190 ceph-mon[31423]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:07.897 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:07 smithi190 ceph-mon[31423]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi169=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:23:07.897 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:07 smithi190 ceph-mon[31423]: Saving service mgr spec with placement smithi026=x;smithi169=y;smithi190=z;count:3 2024-04-21T10:23:07.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:07 smithi026 ceph-mon[27965]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:07.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:07 smithi026 ceph-mon[27965]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi169=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:23:07.919 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:07 smithi026 ceph-mon[27965]: Saving service mgr spec with placement smithi026=x;smithi169=y;smithi190=z;count:3 2024-04-21T10:23:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-04-21T10:23:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-04-21T10:23:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000119785 s, 4.3 MB/s 2024-04-21T10:23:07.922 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T10:23:07.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:07 smithi169 ceph-mon[31469]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:07.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:07 smithi169 ceph-mon[31469]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi169=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:23:07.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:07 smithi169 ceph-mon[31469]: Saving service mgr spec with placement smithi026=x;smithi169=y;smithi190=z;count:3 2024-04-21T10:23:07.980 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-04-21T10:23:08.037 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T10:23:08.037 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T10:23:08.038 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-21T10:23:08.038 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:23:08.038 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T10:23:08.038 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-21 10:22:12.076070409 +0000 2024-04-21T10:23:08.038 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-21 10:17:22.762687855 +0000 2024-04-21T10:23:08.038 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-21 10:17:22.762687855 +0000 2024-04-21T10:23:08.039 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-21 10:17:22.762687855 +0000 2024-04-21T10:23:08.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T10:23:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-04-21T10:23:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-04-21T10:23:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000112334 s, 4.6 MB/s 2024-04-21T10:23:08.103 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T10:23:08.159 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-04-21T10:23:08.214 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T10:23:08.214 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T10:23:08.214 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-21T10:23:08.214 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:23:08.214 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T10:23:08.215 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-21 10:22:12.104069994 +0000 2024-04-21T10:23:08.215 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-21 10:17:23.102682397 +0000 2024-04-21T10:23:08.215 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-21 10:17:23.102682397 +0000 2024-04-21T10:23:08.215 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-21 10:17:23.102682397 +0000 2024-04-21T10:23:08.215 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T10:23:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-04-21T10:23:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-04-21T10:23:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000104663 s, 4.9 MB/s 2024-04-21T10:23:08.279 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T10:23:08.334 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-21T10:23:08.334 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T10:23:08.353 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T10:23:08.353 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-04-21T10:23:08.411 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T10:23:08.411 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T10:23:08.412 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-21T10:23:08.412 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:23:08.412 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T10:23:08.412 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-21 10:22:53.505744321 +0000 2024-04-21T10:23:08.412 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-21 10:17:40.967309428 +0000 2024-04-21T10:23:08.412 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-21 10:17:40.967309428 +0000 2024-04-21T10:23:08.412 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-21 10:17:40.967309428 +0000 2024-04-21T10:23:08.412 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T10:23:08.477 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-21T10:23:08.478 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-21T10:23:08.478 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000289696 s, 1.8 MB/s 2024-04-21T10:23:08.479 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T10:23:08.537 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-04-21T10:23:08.594 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T10:23:08.594 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T10:23:08.594 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-21T10:23:08.594 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:23:08.595 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T10:23:08.595 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-21 10:22:53.506744299 +0000 2024-04-21T10:23:08.595 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-21 10:17:41.311301629 +0000 2024-04-21T10:23:08.595 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-21 10:17:41.311301629 +0000 2024-04-21T10:23:08.595 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-21 10:17:41.311301629 +0000 2024-04-21T10:23:08.595 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T10:23:08.659 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-21T10:23:08.660 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-21T10:23:08.660 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000274491 s, 1.9 MB/s 2024-04-21T10:23:08.661 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T10:23:08.718 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-04-21T10:23:08.774 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T10:23:08.775 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T10:23:08.775 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-21T10:23:08.775 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:23:08.775 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T10:23:08.775 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-21 10:22:53.506744299 +0000 2024-04-21T10:23:08.775 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-21 10:17:41.628294443 +0000 2024-04-21T10:23:08.775 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-21 10:17:41.628294443 +0000 2024-04-21T10:23:08.775 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-21 10:17:41.628294443 +0000 2024-04-21T10:23:08.775 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T10:23:08.838 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-21T10:23:08.838 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-21T10:23:08.839 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000172927 s, 3.0 MB/s 2024-04-21T10:23:08.839 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T10:23:08.896 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-04-21T10:23:08.952 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T10:23:08.952 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T10:23:08.952 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-21T10:23:08.952 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:23:08.952 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T10:23:08.952 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-21 10:22:53.506744299 +0000 2024-04-21T10:23:08.952 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-21 10:17:41.955287030 +0000 2024-04-21T10:23:08.952 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-21 10:17:41.955287030 +0000 2024-04-21T10:23:08.952 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-21 10:17:41.955287030 +0000 2024-04-21T10:23:08.953 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T10:23:09.015 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-21T10:23:09.015 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-21T10:23:09.015 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000202049 s, 2.5 MB/s 2024-04-21T10:23:09.016 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T10:23:09.072 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-21T10:23:09.072 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T10:23:09.088 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T10:23:09.088 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-04-21T10:23:09.144 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T10:23:09.144 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T10:23:09.144 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-21T10:23:09.144 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:23:09.144 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T10:23:09.144 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-21 10:22:43.776921934 +0000 2024-04-21T10:23:09.144 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-21 10:17:53.301517501 +0000 2024-04-21T10:23:09.144 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-21 10:17:53.301517501 +0000 2024-04-21T10:23:09.144 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-21 10:17:53.301517501 +0000 2024-04-21T10:23:09.145 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T10:23:09.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:09.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:09.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:09.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:09 smithi169 ceph-mon[31469]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:09.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:09.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:09.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:09.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T10:23:09.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T10:23:09.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T10:23:09.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:09.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:09 smithi169 ceph-mon[31469]: Deploying daemon mgr.z on smithi190 2024-04-21T10:23:09.206 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-04-21T10:23:09.207 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-04-21T10:23:09.207 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000266815 s, 1.9 MB/s 2024-04-21T10:23:09.208 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T10:23:09.264 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-04-21T10:23:09.268 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:09.268 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:09.268 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:09.268 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:09 smithi190 ceph-mon[31423]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:09.268 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:09.268 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:09.268 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:09.268 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T10:23:09.269 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T10:23:09.269 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T10:23:09.269 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:09.269 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:09 smithi190 ceph-mon[31423]: Deploying daemon mgr.z on smithi190 2024-04-21T10:23:09.282 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T10:23:09.282 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T10:23:09.282 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-21T10:23:09.282 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:23:09.282 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T10:23:09.282 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-21 10:22:43.776921934 +0000 2024-04-21T10:23:09.283 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-21 10:17:53.665507104 +0000 2024-04-21T10:23:09.283 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-21 10:17:53.665507104 +0000 2024-04-21T10:23:09.283 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-21 10:17:53.665507104 +0000 2024-04-21T10:23:09.283 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T10:23:09.344 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-04-21T10:23:09.345 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-04-21T10:23:09.345 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000223248 s, 2.3 MB/s 2024-04-21T10:23:09.346 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T10:23:09.401 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-04-21T10:23:09.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:09.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:09.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:09.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:09 smithi026 ceph-mon[27965]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:09.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:09.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:09.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:09.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T10:23:09.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T10:23:09.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T10:23:09.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:09.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:09 smithi026 ceph-mon[27965]: Deploying daemon mgr.z on smithi190 2024-04-21T10:23:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T10:23:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T10:23:09.456 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-21T10:23:09.456 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:23:09.456 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T10:23:09.456 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-21 10:22:43.777921906 +0000 2024-04-21T10:23:09.456 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-21 10:17:54.002497477 +0000 2024-04-21T10:23:09.456 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-21 10:17:54.002497477 +0000 2024-04-21T10:23:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-21 10:17:54.002497477 +0000 2024-04-21T10:23:09.457 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T10:23:09.518 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-04-21T10:23:09.519 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-04-21T10:23:09.519 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000246084 s, 2.1 MB/s 2024-04-21T10:23:09.520 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T10:23:09.577 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-04-21T10:23:09.632 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T10:23:09.632 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T10:23:09.632 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-21T10:23:09.632 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:23:09.632 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T10:23:09.632 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-21 10:22:43.777921906 +0000 2024-04-21T10:23:09.632 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-21 10:17:54.337487907 +0000 2024-04-21T10:23:09.632 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-21 10:17:54.337487907 +0000 2024-04-21T10:23:09.632 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-21 10:17:54.337487907 +0000 2024-04-21T10:23:09.633 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T10:23:09.694 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-04-21T10:23:09.694 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-04-21T10:23:09.694 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000207079 s, 2.5 MB/s 2024-04-21T10:23:09.695 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T10:23:09.751 INFO:tasks.cephadm:Deploying osd.0 on smithi026 with /dev/vg_nvme/lv_4... 2024-04-21T10:23:09.751 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T10:23:10.002 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:23:10.919 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:10 smithi190 systemd[1]: Starting Ceph mgr.z for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:23:11.189 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:11 smithi190 ceph-mon[31423]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:11.189 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:10 smithi190 podman[33279]: 2024-04-21 10:23:10.822955618 +0000 UTC m=+0.017923308 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:23:11.189 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:10 smithi190 podman[33279]: 2024-04-21 10:23:10.92939187 +0000 UTC m=+0.124359540 container create 189f62033133a8464ba88d15fcf061c150b7f15b1a070ca1d893906627cbe52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-21T10:23:11.190 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:11 smithi190 podman[33279]: 2024-04-21 10:23:11.106649373 +0000 UTC m=+0.301617045 container init 189f62033133a8464ba88d15fcf061c150b7f15b1a070ca1d893906627cbe52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True) 2024-04-21T10:23:11.190 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:11 smithi190 podman[33279]: 2024-04-21 10:23:11.109137073 +0000 UTC m=+0.304104741 container start 189f62033133a8464ba88d15fcf061c150b7f15b1a070ca1d893906627cbe52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0) 2024-04-21T10:23:11.190 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:11 smithi190 ceph-mgr[33298]: -- 172.21.15.190:0/793063599 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f24af65a0 con 0x562f24acb000 2024-04-21T10:23:11.190 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:11 smithi190 bash[33279]: 189f62033133a8464ba88d15fcf061c150b7f15b1a070ca1d893906627cbe52f 2024-04-21T10:23:11.352 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:11 smithi026 ceph-mon[27965]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:11.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:11 smithi169 ceph-mon[31469]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:11.452 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:11 smithi190 systemd[1]: Started Ceph mgr.z for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:23:11.452 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:11 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:11.266+0000 7f509e5bd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T10:23:11.452 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:11 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:11.452+0000 7f509e5bd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:23:11.776 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:11 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:11.554+0000 7f509e5bd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T10:23:11.918 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T10:23:11.918 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T10:23:11.918 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-04-21T10:23:11.918 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-04-21T10:23:11.918 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00832245 s, 1.3 GB/s 2024-04-21T10:23:11.918 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-04-21T10:23:12.261 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:11 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:11.861+0000 7f509e5bd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T10:23:12.261 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:11 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:11.945+0000 7f509e5bd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T10:23:12.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:12.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:12.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:12.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T10:23:12.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T10:23:12.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T10:23:12.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:12.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:12 smithi169 ceph-mon[31469]: Deploying daemon mgr.y on smithi169 2024-04-21T10:23:12.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:12.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:12.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:12.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T10:23:12.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T10:23:12.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T10:23:12.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:12.488 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:12 smithi026 ceph-mon[27965]: Deploying daemon mgr.y on smithi169 2024-04-21T10:23:12.516 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_4 2024-04-21T10:23:12.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:12.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:12.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:12.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T10:23:12.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T10:23:12.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T10:23:12.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:12.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:12 smithi190 ceph-mon[31423]: Deploying daemon mgr.y on smithi169 2024-04-21T10:23:12.757 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:23:13.282 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:13 smithi169 systemd[1]: Starting Ceph mgr.y for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:23:13.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:13 smithi026 ceph-mon[27965]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:13.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:13 smithi190 ceph-mon[31423]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:13.495 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:13 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:13.163+0000 7f509e5bd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T10:23:13.495 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:13 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:13.333+0000 7f509e5bd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T10:23:13.585 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:13 smithi169 ceph-mon[31469]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:13.586 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:13 smithi169 podman[32845]: 2024-04-21 10:23:13.373967373 +0000 UTC m=+0.017820679 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:23:13.586 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:13 smithi169 podman[32845]: 2024-04-21 10:23:13.48191195 +0000 UTC m=+0.125765253 container create d677f561df8a4493485e27feaafa89e9911153540d9f963954afea6794dc0bef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:23:13.777 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:13 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:13.516+0000 7f509e5bd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T10:23:13.948 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:13 smithi169 podman[32845]: 2024-04-21 10:23:13.692378642 +0000 UTC m=+0.336231945 container init d677f561df8a4493485e27feaafa89e9911153540d9f963954afea6794dc0bef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-21T10:23:13.948 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:13 smithi169 podman[32845]: 2024-04-21 10:23:13.695807593 +0000 UTC m=+0.339660898 container start d677f561df8a4493485e27feaafa89e9911153540d9f963954afea6794dc0bef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-21T10:23:13.948 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:13 smithi169 ceph-mgr[32864]: -- 172.21.15.169:0/1472010396 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b8075a5a0 con 0x560b8072f000 2024-04-21T10:23:13.948 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:13 smithi169 bash[32845]: d677f561df8a4493485e27feaafa89e9911153540d9f963954afea6794dc0bef 2024-04-21T10:23:13.948 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:13 smithi169 systemd[1]: Started Ceph mgr.y for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:23:13.948 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:13 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:13.848+0000 7f59ebb3e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T10:23:14.095 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:13 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:13.797+0000 7f509e5bd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T10:23:14.095 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:13 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:13.889+0000 7f509e5bd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T10:23:14.095 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:14 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:14.095+0000 7f509e5bd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T10:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.295+0000 7f6f5ee41700 1 -- 172.21.15.26:0/1418752133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f58102ba0 msgr2=0x7f6f58103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.295+0000 7f6f5ee41700 1 --2- 172.21.15.26:0/1418752133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f58102ba0 0x7f6f58103000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0099c0 tx=0x7f6f4c009cd0 comp rx=0 tx=0).stop 2024-04-21T10:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.295+0000 7f6f5ee41700 1 -- 172.21.15.26:0/1418752133 shutdown_connections 2024-04-21T10:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.295+0000 7f6f5ee41700 1 --2- 172.21.15.26:0/1418752133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f581019a0 0x7f6f58101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.295+0000 7f6f5ee41700 1 --2- 172.21.15.26:0/1418752133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f58103540 0x7f6f58109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.295+0000 7f6f5ee41700 1 --2- 172.21.15.26:0/1418752133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f58102ba0 0x7f6f58103000 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.295+0000 7f6f5ee41700 1 -- 172.21.15.26:0/1418752133 >> 172.21.15.26:0/1418752133 conn(0x7f6f580fcf40 msgr2=0x7f6f580ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.295+0000 7f6f5ee41700 1 -- 172.21.15.26:0/1418752133 shutdown_connections 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.295+0000 7f6f5ee41700 1 -- 172.21.15.26:0/1418752133 wait complete. 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f5ee41700 1 Processor -- start 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f5ee41700 1 -- start start 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f5ee41700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f581019a0 0x7f6f5819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f5ee41700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f58102ba0 0x7f6f5819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f5ee41700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f58103540 0x7f6f581a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f5ee41700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5810d490 con 0x7f6f58102ba0 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f5ee41700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5810d310 con 0x7f6f58103540 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f5ee41700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5810d610 con 0x7f6f581019a0 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f57fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f58102ba0 0x7f6f5819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f57fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f58102ba0 0x7f6f5819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40208/0 (socket says 172.21.15.26:40208) 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f57fff700 1 -- 172.21.15.26:0/589331915 learned_addr learned my addr 172.21.15.26:0/589331915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f5cbdd700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f581019a0 0x7f6f5819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.296+0000 7f6f5d3de700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f58103540 0x7f6f581a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.297+0000 7f6f57fff700 1 -- 172.21.15.26:0/589331915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f581019a0 msgr2=0x7f6f5819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.297+0000 7f6f57fff700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f581019a0 0x7f6f5819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.297+0000 7f6f57fff700 1 -- 172.21.15.26:0/589331915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f58103540 msgr2=0x7f6f581a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.297+0000 7f6f57fff700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f58103540 0x7f6f581a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.297+0000 7f6f57fff700 1 -- 172.21.15.26:0/589331915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f581a2ab0 con 0x7f6f58102ba0 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.297+0000 7f6f5cbdd700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f581019a0 0x7f6f5819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.297+0000 7f6f57fff700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f58102ba0 0x7f6f5819b400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00f690 tx=0x7f6f4c00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.297+0000 7f6f55ffb700 1 -- 172.21.15.26:0/589331915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c009e60 con 0x7f6f58102ba0 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.297+0000 7f6f55ffb700 1 -- 172.21.15.26:0/589331915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f4c0038c0 con 0x7f6f58102ba0 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.297+0000 7f6f5ee41700 1 -- 172.21.15.26:0/589331915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f581a2d40 con 0x7f6f58102ba0 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.297+0000 7f6f55ffb700 1 -- 172.21.15.26:0/589331915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c005670 con 0x7f6f58102ba0 2024-04-21T10:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.297+0000 7f6f5ee41700 1 -- 172.21.15.26:0/589331915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f581a31a0 con 0x7f6f58102ba0 2024-04-21T10:23:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.298+0000 7f6f55ffb700 1 -- 172.21.15.26:0/589331915 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6f4c02b440 con 0x7f6f58102ba0 2024-04-21T10:23:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.298+0000 7f6f5ee41700 1 -- 172.21.15.26:0/589331915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f58069730 con 0x7f6f58102ba0 2024-04-21T10:23:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.298+0000 7f6f55ffb700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6f40039890 0x7f6f4003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.298+0000 7f6f55ffb700 1 -- 172.21.15.26:0/589331915 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6f4c04d810 con 0x7f6f58102ba0 2024-04-21T10:23:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.298+0000 7f6f5cbdd700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6f40039890 0x7f6f4003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.299+0000 7f6f5cbdd700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6f40039890 0x7f6f4003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f58102a00 tx=0x7f6f48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:23:14.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.303+0000 7f6f55ffb700 1 -- 172.21.15.26:0/589331915 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f4c013070 con 0x7f6f58102ba0 2024-04-21T10:23:14.448 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:14.036+0000 7f59ebb3e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:23:14.448 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:14.145+0000 7f59ebb3e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T10:23:14.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:14.520+0000 7f6f5ee41700 1 -- 172.21.15.26:0/589331915 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6f580ff120 con 0x7f6f40039890 2024-04-21T10:23:14.526 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:14 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:14.293+0000 7f509e5bd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:23:14.842 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:14.464+0000 7f59ebb3e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T10:23:14.842 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:14.563+0000 7f59ebb3e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T10:23:15.150 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:15.150 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:15.150 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:15.150 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:15.150 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:15.150 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:23:15.150 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:23:15.150 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:14 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:14 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:14 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:14 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:14 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:14 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:23:15.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:14 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:23:15.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:14 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:15.277 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:14 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:14.915+0000 7f509e5bd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T10:23:15.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:14 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:15.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:14 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:15.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:14 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:15.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:14 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:15.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:14 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:15.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:14 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:23:15.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:14 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:23:15.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:14 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:15.658 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:15.363+0000 7f509e5bd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T10:23:15.659 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:15.459+0000 7f509e5bd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T10:23:15.659 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:15.659+0000 7f509e5bd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T10:23:16.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-mon[31423]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:16.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-mon[31423]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:23:16.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:16.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:16.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T10:23:16.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T10:23:16.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:16.069 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:15 smithi026 ceph-mon[27965]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:16.069 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:15 smithi026 ceph-mon[27965]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:23:16.069 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.069 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.069 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.069 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.069 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:16.069 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:16.069 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.069 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T10:23:16.070 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T10:23:16.070 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:16.167 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:15.824+0000 7f59ebb3e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T10:23:16.167 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:15.995+0000 7f59ebb3e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T10:23:16.167 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:16 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:16.167+0000 7f59ebb3e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T10:23:16.167 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-mon[31469]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:16.167 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-mon[31469]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:23:16.167 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.167 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.168 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.168 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.168 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:16.168 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:16.168 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:16.168 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T10:23:16.168 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T10:23:16.168 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:16.498 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:16 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:16.175+0000 7f509e5bd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T10:23:16.498 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:16 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:16.264+0000 7f509e5bd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T10:23:16.746 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:16 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:16.452+0000 7f59ebb3e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T10:23:16.746 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:16 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:16.543+0000 7f59ebb3e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T10:23:16.746 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:16 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:16.746+0000 7f59ebb3e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T10:23:16.777 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:16 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:16.497+0000 7f509e5bd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T10:23:16.777 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:16 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:16.627+0000 7f509e5bd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T10:23:16.777 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:16 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:16.718+0000 7f509e5bd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T10:23:17.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:16 smithi190 ceph-mon[31423]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T10:23:17.132 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:16 smithi190 ceph-mon[31423]: Reconfiguring daemon mgr.x on smithi026 2024-04-21T10:23:17.132 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:16 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:16.830+0000 7f509e5bd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T10:23:17.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:16 smithi026 ceph-mon[27965]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T10:23:17.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:16 smithi026 ceph-mon[27965]: Reconfiguring daemon mgr.x on smithi026 2024-04-21T10:23:17.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:16 smithi169 ceph-mon[31469]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T10:23:17.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:16 smithi169 ceph-mon[31469]: Reconfiguring daemon mgr.x on smithi026 2024-04-21T10:23:17.198 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:16 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:16.938+0000 7f59ebb3e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:23:17.527 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:17 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:17.131+0000 7f509e5bd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T10:23:17.862 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:17 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:17.533+0000 7f59ebb3e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T10:23:18.027 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:17 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:17.720+0000 7f509e5bd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T10:23:18.027 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:23:17 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:23:17.930+0000 7f509e5bd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T10:23:18.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:17 smithi190 ceph-mon[31423]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:18.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:17 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:18.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:17 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1459226705' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a8bc52a-346f-4a31-aad9-535e551ba41b"}]: dispatch 2024-04-21T10:23:18.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:17 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1459226705' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a8bc52a-346f-4a31-aad9-535e551ba41b"}]': finished 2024-04-21T10:23:18.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:17 smithi190 ceph-mon[31423]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T10:23:18.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:17 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:18.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:17 smithi026 ceph-mon[27965]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:18.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:17 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:18.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:17 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1459226705' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a8bc52a-346f-4a31-aad9-535e551ba41b"}]: dispatch 2024-04-21T10:23:18.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:17 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1459226705' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a8bc52a-346f-4a31-aad9-535e551ba41b"}]': finished 2024-04-21T10:23:18.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:17 smithi026 ceph-mon[27965]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T10:23:18.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:17 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:18.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:17 smithi169 ceph-mon[31469]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:18.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:17 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:18.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:17 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1459226705' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a8bc52a-346f-4a31-aad9-535e551ba41b"}]: dispatch 2024-04-21T10:23:18.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:17 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1459226705' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a8bc52a-346f-4a31-aad9-535e551ba41b"}]': finished 2024-04-21T10:23:18.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:17 smithi169 ceph-mon[31469]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T10:23:18.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:17 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:18.199 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:17 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:17.946+0000 7f59ebb3e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T10:23:18.199 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:18 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:18.039+0000 7f59ebb3e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T10:23:18.698 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:18 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:18.237+0000 7f59ebb3e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T10:23:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:18.871+0000 7f6f55ffb700 1 -- 172.21.15.26:0/589331915 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f6f4c0213c0 con 0x7f6f58102ba0 2024-04-21T10:23:19.082 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:18 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3691993182' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:23:19.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:18 smithi169 ceph-mon[31469]: Standby manager daemon z started 2024-04-21T10:23:19.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:18 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.190:0/1524532358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T10:23:19.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:18 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.190:0/1524532358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T10:23:19.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:18 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.190:0/1524532358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T10:23:19.083 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:18 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.190:0/1524532358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T10:23:19.083 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:18 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:18.754+0000 7f59ebb3e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T10:23:19.083 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:18 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:18.844+0000 7f59ebb3e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T10:23:19.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:18 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3691993182' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:23:19.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:18 smithi026 ceph-mon[27965]: Standby manager daemon z started 2024-04-21T10:23:19.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:18 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.190:0/1524532358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T10:23:19.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:18 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.190:0/1524532358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T10:23:19.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:18 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.190:0/1524532358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T10:23:19.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:18 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.190:0/1524532358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T10:23:19.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:18 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3691993182' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:23:19.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:18 smithi190 ceph-mon[31423]: Standby manager daemon z started 2024-04-21T10:23:19.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:18 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.190:0/1524532358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T10:23:19.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:18 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.190:0/1524532358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T10:23:19.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:18 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.190:0/1524532358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T10:23:19.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:18 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.190:0/1524532358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T10:23:19.417 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:19 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:19.082+0000 7f59ebb3e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T10:23:19.417 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:19 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:19.212+0000 7f59ebb3e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T10:23:19.417 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:19 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:19.305+0000 7f59ebb3e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T10:23:19.417 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:19 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:19.417+0000 7f59ebb3e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T10:23:20.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:19 smithi026 ceph-mon[27965]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:20.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:19 smithi026 ceph-mon[27965]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-21T10:23:20.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:19 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T10:23:20.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:19 smithi169 ceph-mon[31469]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:20.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:19 smithi169 ceph-mon[31469]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-21T10:23:20.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:19 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T10:23:20.198 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:19 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:19.717+0000 7f59ebb3e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T10:23:20.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:19 smithi190 ceph-mon[31423]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:20.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:19 smithi190 ceph-mon[31423]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-21T10:23:20.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:19 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T10:23:20.698 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:20 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:20.309+0000 7f59ebb3e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T10:23:20.698 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:23:20 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:23:20.519+0000 7f59ebb3e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T10:23:20.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:20.880+0000 7f6f55ffb700 1 -- 172.21.15.26:0/589331915 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f6f4c0173f0 con 0x7f6f58102ba0 2024-04-21T10:23:21.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:20 smithi026 ceph-mon[27965]: Standby manager daemon y started 2024-04-21T10:23:21.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:20 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.169:0/4220973479' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T10:23:21.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:20 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.169:0/4220973479' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T10:23:21.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:20 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.169:0/4220973479' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T10:23:21.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:20 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.169:0/4220973479' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T10:23:21.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:20 smithi169 ceph-mon[31469]: Standby manager daemon y started 2024-04-21T10:23:21.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:20 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.169:0/4220973479' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T10:23:21.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:20 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.169:0/4220973479' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T10:23:21.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:20 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.169:0/4220973479' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T10:23:21.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:20 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.169:0/4220973479' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T10:23:21.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:20 smithi190 ceph-mon[31423]: Standby manager daemon y started 2024-04-21T10:23:21.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:20 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.169:0/4220973479' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T10:23:21.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:20 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.169:0/4220973479' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T10:23:21.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:20 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.169:0/4220973479' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T10:23:21.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:20 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.169:0/4220973479' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T10:23:22.168 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:21 smithi169 ceph-mon[31469]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:22.168 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:21 smithi169 ceph-mon[31469]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-04-21T10:23:22.168 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:21 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T10:23:22.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:21 smithi026 ceph-mon[27965]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:22.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:21 smithi026 ceph-mon[27965]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-04-21T10:23:22.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:21 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T10:23:22.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:21 smithi190 ceph-mon[31423]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:22.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:21 smithi190 ceph-mon[31423]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-04-21T10:23:22.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:21 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T10:23:23.730 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:23 smithi026 ceph-mon[27965]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:23.730 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:23.730 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:23.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:23.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:23.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:23.731 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:23.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:23 smithi169 ceph-mon[31469]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:23.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:23.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:23.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:23.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:23.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:23.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:23 smithi190 ceph-mon[31423]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:25.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:25 smithi169 ceph-mon[31469]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:25.950 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:25 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T10:23:25.950 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:25 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:25.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:25 smithi026 ceph-mon[27965]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:25.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:25 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T10:23:25.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:25 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:26.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:25 smithi190 ceph-mon[31423]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:26.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:25 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T10:23:26.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:25 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:26.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:26 smithi169 ceph-mon[31469]: Deploying daemon osd.0 on smithi026 2024-04-21T10:23:26.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:26 smithi026 ceph-mon[27965]: Deploying daemon osd.0 on smithi026 2024-04-21T10:23:27.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:26 smithi190 ceph-mon[31423]: Deploying daemon osd.0 on smithi026 2024-04-21T10:23:27.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:27 smithi169 ceph-mon[31469]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:28.022 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:27 smithi026 ceph-mon[27965]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:28.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:27 smithi190 ceph-mon[31423]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:29 smithi026 ceph-mon[27965]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:29 smithi190 ceph-mon[31423]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:30.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:29 smithi169 ceph-mon[31469]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:31 smithi026 ceph-mon[27965]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:31.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:31.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:31.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:31 smithi169 ceph-mon[31469]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:31.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:31 smithi190 ceph-mon[31423]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:31.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:31.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:31.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:33 smithi026 ceph-mon[27965]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:33 smithi169 ceph-mon[31469]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:33 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:33 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:33 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:33 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:33 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:33.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:33 smithi190 ceph-mon[31423]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:33.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:33 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:33.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:33 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:33.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:33 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:33.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:33 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:33.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:33 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:34.276 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 0 on host 'smithi026' 2024-04-21T10:23:34.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.276+0000 7f6f55ffb700 1 -- 172.21.15.26:0/589331915 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6f580ff120 con 0x7f6f40039890 2024-04-21T10:23:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.278+0000 7f6f5ee41700 1 -- 172.21.15.26:0/589331915 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6f40039890 msgr2=0x7f6f4003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.278+0000 7f6f5ee41700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6f40039890 0x7f6f4003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f58102a00 tx=0x7f6f48009260 comp rx=0 tx=0).stop 2024-04-21T10:23:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.278+0000 7f6f5ee41700 1 -- 172.21.15.26:0/589331915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f58102ba0 msgr2=0x7f6f5819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.278+0000 7f6f5ee41700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f58102ba0 0x7f6f5819b400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00f690 tx=0x7f6f4c00f6c0 comp rx=0 tx=0).stop 2024-04-21T10:23:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.278+0000 7f6f5ee41700 1 -- 172.21.15.26:0/589331915 shutdown_connections 2024-04-21T10:23:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.278+0000 7f6f5ee41700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6f40039890 0x7f6f4003bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.278+0000 7f6f5ee41700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f58103540 0x7f6f581a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.278+0000 7f6f5ee41700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f581019a0 0x7f6f5819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.278+0000 7f6f5ee41700 1 --2- 172.21.15.26:0/589331915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f58102ba0 0x7f6f5819b400 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.278+0000 7f6f5ee41700 1 -- 172.21.15.26:0/589331915 >> 172.21.15.26:0/589331915 conn(0x7f6f580fcf40 msgr2=0x7f6f580fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:23:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.279+0000 7f6f5ee41700 1 -- 172.21.15.26:0/589331915 shutdown_connections 2024-04-21T10:23:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:23:34.279+0000 7f6f5ee41700 1 -- 172.21.15.26:0/589331915 wait complete. 2024-04-21T10:23:34.913 DEBUG:teuthology.orchestra.run.smithi026:osd.0> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.0.service 2024-04-21T10:23:34.916 INFO:tasks.cephadm:Deploying osd.1 on smithi169 with /dev/vg_nvme/lv_4... 2024-04-21T10:23:34.916 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T10:23:35.160 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T10:23:35.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:35 smithi026 ceph-mon[27965]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:35.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:35 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:35.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:35 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:35.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:35 smithi026 ceph-mon[27965]: from='osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T10:23:35.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:35 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:35.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:35 smithi190 ceph-mon[31423]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:35.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:35 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:35.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:35 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:35.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:35 smithi190 ceph-mon[31423]: from='osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T10:23:35.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:35 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:35.596 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:35 smithi169 ceph-mon[31469]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:35.596 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:35 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:35.596 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:35 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:35.596 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:35 smithi169 ceph-mon[31469]: from='osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T10:23:35.596 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:35 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:36.392 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:36 smithi169 ceph-mon[31469]: from='osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T10:23:36.393 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:36 smithi169 ceph-mon[31469]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T10:23:36.393 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:36 smithi169 ceph-mon[31469]: from='osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-04-21T10:23:36.393 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:36 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:36.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:36 smithi026 ceph-mon[27965]: from='osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T10:23:36.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:36 smithi026 ceph-mon[27965]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T10:23:36.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:36 smithi026 ceph-mon[27965]: from='osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-04-21T10:23:36.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:36 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:36.436 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:23:36 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0[39885]: 2024-04-21T10:23:36.269+0000 7fd673edb700 -1 osd.0 0 waiting for initial osdmap 2024-04-21T10:23:36.436 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:23:36 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0[39885]: 2024-04-21T10:23:36.272+0000 7fd66fcce700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T10:23:36.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:36 smithi190 ceph-mon[31423]: from='osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T10:23:36.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:36 smithi190 ceph-mon[31423]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T10:23:36.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:36 smithi190 ceph-mon[31423]: from='osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-04-21T10:23:36.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:36 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T10:23:36.960 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131506 s, 797 MB/s 2024-04-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 2024-04-21T10:23:36.961 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-21T10:23:37.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:37 smithi190 ceph-mon[31423]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:37.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:37 smithi190 ceph-mon[31423]: from='osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-04-21T10:23:37.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:37 smithi190 ceph-mon[31423]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T10:23:37.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:37.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:37.532 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:37 smithi169 ceph-mon[31469]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:37.532 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:37 smithi169 ceph-mon[31469]: from='osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-04-21T10:23:37.532 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:37 smithi169 ceph-mon[31469]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T10:23:37.532 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:37.532 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:37.560 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_4 2024-04-21T10:23:37.592 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:37 smithi026 ceph-mon[27965]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T10:23:37.592 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:37 smithi026 ceph-mon[27965]: from='osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-04-21T10:23:37.592 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:37 smithi026 ceph-mon[27965]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T10:23:37.592 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:37.592 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:37.803 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T10:23:38.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: purged_snaps scrub starts 2024-04-21T10:23:38.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: purged_snaps scrub ok 2024-04-21T10:23:38.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108] boot 2024-04-21T10:23:38.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T10:23:38.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:38.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: Detected new or changed devices on smithi026 2024-04-21T10:23:38.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:38.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:38.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:23:38.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi026 to 17093M 2024-04-21T10:23:38.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:38.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:38.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:38.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:38 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:38.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: purged_snaps scrub starts 2024-04-21T10:23:38.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: purged_snaps scrub ok 2024-04-21T10:23:38.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108] boot 2024-04-21T10:23:38.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T10:23:38.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:38.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: Detected new or changed devices on smithi026 2024-04-21T10:23:38.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:38.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:38.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:23:38.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi026 to 17093M 2024-04-21T10:23:38.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:38.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:38.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:38.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:38 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:38.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: purged_snaps scrub starts 2024-04-21T10:23:38.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: purged_snaps scrub ok 2024-04-21T10:23:38.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: osd.0 [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108] boot 2024-04-21T10:23:38.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T10:23:38.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:23:38.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: Detected new or changed devices on smithi026 2024-04-21T10:23:38.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:38.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:38.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:23:38.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi026 to 17093M 2024-04-21T10:23:38.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:38.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:38.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:38.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.357+0000 7f0740c0d700 1 -- 172.21.15.169:0/1725953615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f073c104670 msgr2=0x7f073c10af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.357+0000 7f0740c0d700 1 --2- 172.21.15.169:0/1725953615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f073c104670 0x7f073c10af20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f073400b0d0 tx=0x7f073400b3e0 comp rx=0 tx=0).stop 2024-04-21T10:23:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.357+0000 7f0740c0d700 1 -- 172.21.15.169:0/1725953615 shutdown_connections 2024-04-21T10:23:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.357+0000 7f0740c0d700 1 --2- 172.21.15.169:0/1725953615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f073c104670 0x7f073c10af20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.357+0000 7f0740c0d700 1 --2- 172.21.15.169:0/1725953615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f073c102ad0 0x7f073c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.357+0000 7f0740c0d700 1 --2- 172.21.15.169:0/1725953615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f073c103cd0 0x7f073c104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.357+0000 7f0740c0d700 1 -- 172.21.15.169:0/1725953615 >> 172.21.15.169:0/1725953615 conn(0x7f073c0fe090 msgr2=0x7f073c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:23:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.357+0000 7f0740c0d700 1 -- 172.21.15.169:0/1725953615 shutdown_connections 2024-04-21T10:23:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.357+0000 7f0740c0d700 1 -- 172.21.15.169:0/1725953615 wait complete. 2024-04-21T10:23:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.357+0000 7f0740c0d700 1 Processor -- start 2024-04-21T10:23:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f0740c0d700 1 -- start start 2024-04-21T10:23:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f0740c0d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f073c102ad0 0x7f073c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f0740c0d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f073c103cd0 0x7f073c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f0740c0d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f073c104670 0x7f073c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f0740c0d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f073c10e570 con 0x7f073c102ad0 2024-04-21T10:23:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f0740c0d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f073c10e3f0 con 0x7f073c104670 2024-04-21T10:23:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f0740c0d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f073c10e6f0 con 0x7f073c103cd0 2024-04-21T10:23:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f073ad9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f073c104670 0x7f073c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f073ad9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f073c104670 0x7f073c075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57104/0 (socket says 172.21.15.169:57104) 2024-04-21T10:23:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f073ad9d700 1 -- 172.21.15.169:0/1787413945 learned_addr learned my addr 172.21.15.169:0/1787413945 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f073a59c700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f073c102ad0 0x7f073c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f0731d9b700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f073c103cd0 0x7f073c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f073ad9d700 1 -- 172.21.15.169:0/1787413945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f073c103cd0 msgr2=0x7f073c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f073ad9d700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f073c103cd0 0x7f073c074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f073ad9d700 1 -- 172.21.15.169:0/1787413945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f073c102ad0 msgr2=0x7f073c0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f073ad9d700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f073c102ad0 0x7f073c0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f073ad9d700 1 -- 172.21.15.169:0/1787413945 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c075c20 con 0x7f073c104670 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.358+0000 7f073ad9d700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f073c104670 0x7f073c075370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0734009fd0 tx=0x7f0734004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.359+0000 7f0733fff700 1 -- 172.21.15.169:0/1787413945 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073401f030 con 0x7f073c104670 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.359+0000 7f0740c0d700 1 -- 172.21.15.169:0/1787413945 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073c1a2c20 con 0x7f073c104670 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.359+0000 7f0733fff700 1 -- 172.21.15.169:0/1787413945 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0734007a80 con 0x7f073c104670 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.359+0000 7f0733fff700 1 -- 172.21.15.169:0/1787413945 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734004810 con 0x7f073c104670 2024-04-21T10:23:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.359+0000 7f0740c0d700 1 -- 172.21.15.169:0/1787413945 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f073c1a3070 con 0x7f073c104670 2024-04-21T10:23:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.360+0000 7f0740c0d700 1 -- 172.21.15.169:0/1787413945 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f073c02df80 con 0x7f073c104670 2024-04-21T10:23:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.361+0000 7f0733fff700 1 -- 172.21.15.169:0/1787413945 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f0734003a00 con 0x7f073c104670 2024-04-21T10:23:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.361+0000 7f0733fff700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f07280a3bb0 0x7f07280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:23:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.361+0000 7f0733fff700 1 -- 172.21.15.169:0/1787413945 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f073400f020 con 0x7f073c104670 2024-04-21T10:23:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.362+0000 7f073a59c700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f07280a3bb0 0x7f07280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:23:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.362+0000 7f073a59c700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f07280a3bb0 0x7f07280a6070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f073c103b30 tx=0x7f072c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:23:39.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.365+0000 7f0733fff700 1 -- 172.21.15.169:0/1787413945 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0734029020 con 0x7f073c104670 2024-04-21T10:23:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:39 smithi169 ceph-mon[31469]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:39 smithi169 ceph-mon[31469]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T10:23:39.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:39 smithi190 ceph-mon[31423]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:39.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:39 smithi190 ceph-mon[31423]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T10:23:39.571 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:39.571+0000 7f0740c0d700 1 -- 172.21.15.169:0/1787413945 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f073c062e80 con 0x7f07280a3bb0 2024-04-21T10:23:39.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:39 smithi026 ceph-mon[27965]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:39.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:39 smithi026 ceph-mon[27965]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T10:23:40.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:40 smithi169 ceph-mon[31469]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:23:40.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:23:40.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:23:40.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:40.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:40 smithi026 ceph-mon[27965]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:23:40.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:23:40.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:23:40.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:40.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:40 smithi190 ceph-mon[31423]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:23:40.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:23:40.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:23:40.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:41.384 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:41 smithi169 ceph-mon[31469]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:41.678 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:41 smithi190 ceph-mon[31423]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:41 smithi026 ceph-mon[27965]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:42.581 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:42 smithi169 ceph-mon[31469]: from='client.? 172.21.15.169:0/221800455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a"}]: dispatch 2024-04-21T10:23:42.581 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:42 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a"}]: dispatch 2024-04-21T10:23:42.581 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:42 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a"}]': finished 2024-04-21T10:23:42.581 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:42 smithi169 ceph-mon[31469]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T10:23:42.581 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:42 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:23:42.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:42 smithi026 ceph-mon[27965]: from='client.? 172.21.15.169:0/221800455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a"}]: dispatch 2024-04-21T10:23:42.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:42 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a"}]: dispatch 2024-04-21T10:23:42.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:42 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a"}]': finished 2024-04-21T10:23:42.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:42 smithi026 ceph-mon[27965]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T10:23:42.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:42 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:23:42.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:42 smithi190 ceph-mon[31423]: from='client.? 172.21.15.169:0/221800455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a"}]: dispatch 2024-04-21T10:23:42.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:42 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a"}]: dispatch 2024-04-21T10:23:42.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:42 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a"}]': finished 2024-04-21T10:23:42.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:42 smithi190 ceph-mon[31423]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T10:23:42.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:42 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:23:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:43 smithi026 ceph-mon[27965]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:43 smithi026 ceph-mon[27965]: from='client.? 172.21.15.169:0/2169808633' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:23:43.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:43 smithi169 ceph-mon[31469]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:43.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:43 smithi169 ceph-mon[31469]: from='client.? 172.21.15.169:0/2169808633' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:23:43.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:43 smithi190 ceph-mon[31423]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:43.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:43 smithi190 ceph-mon[31423]: from='client.? 172.21.15.169:0/2169808633' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:23:45.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:45 smithi026 ceph-mon[27965]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:45.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:45 smithi169 ceph-mon[31469]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:45.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:45 smithi190 ceph-mon[31423]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:47.587 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:47 smithi169 ceph-mon[31469]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:47.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:47 smithi026 ceph-mon[27965]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:47.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:47 smithi190 ceph-mon[31423]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:49.563 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:49 smithi169 ceph-mon[31469]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:49 smithi026 ceph-mon[27965]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:49.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:49 smithi190 ceph-mon[31423]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:50.498 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:50 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T10:23:50.499 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:50 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:50.499 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:50 smithi169 ceph-mon[31469]: Deploying daemon osd.1 on smithi169 2024-04-21T10:23:50.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T10:23:50.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:50.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:50 smithi026 ceph-mon[27965]: Deploying daemon osd.1 on smithi169 2024-04-21T10:23:50.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T10:23:50.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:50.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:50 smithi190 ceph-mon[31423]: Deploying daemon osd.1 on smithi169 2024-04-21T10:23:51.538 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:51 smithi169 ceph-mon[31469]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:51.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:51 smithi026 ceph-mon[27965]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:51.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:51 smithi190 ceph-mon[31423]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:53.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:53 smithi169 ceph-mon[31469]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:53.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:53 smithi026 ceph-mon[27965]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:53.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:53 smithi190 ceph-mon[31423]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:55.628 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:55 smithi169 ceph-mon[31469]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:55.628 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:55 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:55.628 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:55 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:55.628 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:55 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:55.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:55 smithi026 ceph-mon[27965]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:55.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:55 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:55.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:55 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:55.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:55 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:55.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:55 smithi190 ceph-mon[31423]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:55.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:55 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:55.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:55 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:55.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:55 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:57.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:57 smithi169 ceph-mon[31469]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:57.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:57 smithi026 ceph-mon[27965]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:57.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:57 smithi190 ceph-mon[31423]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:58.644 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.645+0000 7f0733fff700 1 -- 172.21.15.169:0/1787413945 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f073c062e80 con 0x7f07280a3bb0 2024-04-21T10:23:58.645 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 1 on host 'smithi169' 2024-04-21T10:23:58.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.647+0000 7f0740c0d700 1 -- 172.21.15.169:0/1787413945 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f07280a3bb0 msgr2=0x7f07280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:58.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.647+0000 7f0740c0d700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f07280a3bb0 0x7f07280a6070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f073c103b30 tx=0x7f072c00a3b0 comp rx=0 tx=0).stop 2024-04-21T10:23:58.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.647+0000 7f0740c0d700 1 -- 172.21.15.169:0/1787413945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f073c104670 msgr2=0x7f073c075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:23:58.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.647+0000 7f0740c0d700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f073c104670 0x7f073c075370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0734009fd0 tx=0x7f0734004e40 comp rx=0 tx=0).stop 2024-04-21T10:23:58.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.647+0000 7f0740c0d700 1 -- 172.21.15.169:0/1787413945 shutdown_connections 2024-04-21T10:23:58.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.647+0000 7f0740c0d700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f07280a3bb0 0x7f07280a6070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:58.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.647+0000 7f0740c0d700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f073c104670 0x7f073c075370 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:58.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.647+0000 7f0740c0d700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f073c102ad0 0x7f073c0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:58.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.647+0000 7f0740c0d700 1 --2- 172.21.15.169:0/1787413945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f073c103cd0 0x7f073c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:23:58.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.647+0000 7f0740c0d700 1 -- 172.21.15.169:0/1787413945 >> 172.21.15.169:0/1787413945 conn(0x7f073c0fe090 msgr2=0x7f073c106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:23:58.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.647+0000 7f0740c0d700 1 -- 172.21.15.169:0/1787413945 shutdown_connections 2024-04-21T10:23:58.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:23:58.647+0000 7f0740c0d700 1 -- 172.21.15.169:0/1787413945 wait complete. 2024-04-21T10:23:58.899 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:58.899 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:58.899 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:58.899 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:58.899 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:58.899 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:58.899 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:58.899 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:59.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:59.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:59.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:59.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:59.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:59.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:59.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:59.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:59.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:59.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:59.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:23:59.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:23:59.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:59.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:59.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:23:59.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:23:59.288 DEBUG:teuthology.orchestra.run.smithi169:osd.1> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.1.service 2024-04-21T10:23:59.290 INFO:tasks.cephadm:Deploying osd.2 on smithi190 with /dev/vg_nvme/lv_4... 2024-04-21T10:23:59.290 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T10:23:59.532 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:23:59.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:59 smithi169 ceph-mon[31469]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:59.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:59 smithi169 ceph-mon[31469]: from='osd.1 [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T10:23:59.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:23:59 smithi169 ceph-mon[31469]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T10:23:59.950 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:59 smithi190 ceph-mon[31423]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:23:59.950 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:59 smithi190 ceph-mon[31423]: from='osd.1 [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T10:23:59.950 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:23:59 smithi190 ceph-mon[31423]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T10:24:00.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:59 smithi026 ceph-mon[27965]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:24:00.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:59 smithi026 ceph-mon[27965]: from='osd.1 [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T10:24:00.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:23:59 smithi026 ceph-mon[27965]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T10:24:00.805 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:00 smithi190 ceph-mon[31423]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T10:24:00.806 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:00 smithi190 ceph-mon[31423]: from='osd.1 [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:24:00.806 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:00 smithi190 ceph-mon[31423]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T10:24:00.806 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:00.806 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:00 smithi190 ceph-mon[31423]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:24:00.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:00 smithi169 ceph-mon[31469]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T10:24:00.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:00 smithi169 ceph-mon[31469]: from='osd.1 [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:24:00.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:00 smithi169 ceph-mon[31469]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T10:24:00.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:00 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:00.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:00 smithi169 ceph-mon[31469]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:24:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:00 smithi026 ceph-mon[27965]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T10:24:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:00 smithi026 ceph-mon[27965]: from='osd.1 [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:24:01.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:00 smithi026 ceph-mon[27965]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T10:24:01.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:01.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:00 smithi026 ceph-mon[27965]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:24:01.371 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T10:24:01.371 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T10:24:01.371 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-04-21T10:24:01.372 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-04-21T10:24:01.372 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115074 s, 911 MB/s 2024-04-21T10:24:01.372 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-04-21T10:24:01.927 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:01 smithi169 ceph-mon[31469]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:24:01.927 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:01 smithi169 ceph-mon[31469]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-21T10:24:01.927 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:01 smithi169 ceph-mon[31469]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T10:24:01.927 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:01.927 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:01.927 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:01 smithi169 ceph-mon[31469]: from='osd.1 ' entity='osd.1' 2024-04-21T10:24:01.927 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:24:01 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1[36066]: 2024-04-21T10:24:01.661+0000 7fe88a38d700 -1 osd.1 0 waiting for initial osdmap 2024-04-21T10:24:01.927 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:24:01 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1[36066]: 2024-04-21T10:24:01.798+0000 7fe881977700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T10:24:01.993 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:01 smithi190 ceph-mon[31423]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:24:01.993 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:01 smithi190 ceph-mon[31423]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-21T10:24:01.993 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:01 smithi190 ceph-mon[31423]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T10:24:01.993 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:01.993 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:01.993 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:01 smithi190 ceph-mon[31423]: from='osd.1 ' entity='osd.1' 2024-04-21T10:24:02.021 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_4 2024-04-21T10:24:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:01 smithi026 ceph-mon[27965]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:24:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:01 smithi026 ceph-mon[27965]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-21T10:24:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:01 smithi026 ceph-mon[27965]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T10:24:02.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:02.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:02.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:01 smithi026 ceph-mon[27965]: from='osd.1 ' entity='osd.1' 2024-04-21T10:24:02.270 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:24:03.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:02 smithi190 ceph-mon[31423]: purged_snaps scrub starts 2024-04-21T10:24:03.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:02 smithi190 ceph-mon[31423]: purged_snaps scrub ok 2024-04-21T10:24:03.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:03.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:03.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:03.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:24:03.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:03.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:03.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:03.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:03.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:02 smithi026 ceph-mon[27965]: purged_snaps scrub starts 2024-04-21T10:24:03.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:02 smithi026 ceph-mon[27965]: purged_snaps scrub ok 2024-04-21T10:24:03.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:03.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:03.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:03.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:24:03.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:03.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:03.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:03.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:03.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:02 smithi169 ceph-mon[31469]: purged_snaps scrub starts 2024-04-21T10:24:03.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:02 smithi169 ceph-mon[31469]: purged_snaps scrub ok 2024-04-21T10:24:03.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:03.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:03.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:03.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:24:03.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:03.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:03.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:03.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:03.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.815+0000 7f9750c44700 1 -- 172.21.15.190:0/3877258936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c103d70 msgr2=0x7f974c1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:03.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.815+0000 7f9750c44700 1 --2- 172.21.15.190:0/3877258936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c103d70 0x7f974c1041d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f973c009990 tx=0x7f973c009ca0 comp rx=0 tx=0).stop 2024-04-21T10:24:03.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.815+0000 7f9750c44700 1 -- 172.21.15.190:0/3877258936 shutdown_connections 2024-04-21T10:24:03.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.815+0000 7f9750c44700 1 --2- 172.21.15.190:0/3877258936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f974c104710 0x7f974c10afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:03.817 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.815+0000 7f9750c44700 1 --2- 172.21.15.190:0/3877258936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f974c100d90 0x7f974c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:03.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.815+0000 7f9750c44700 1 --2- 172.21.15.190:0/3877258936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c103d70 0x7f974c1041d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:03.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.815+0000 7f9750c44700 1 -- 172.21.15.190:0/3877258936 >> 172.21.15.190:0/3877258936 conn(0x7f974c0fc6f0 msgr2=0x7f974c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:24:03.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.815+0000 7f9750c44700 1 -- 172.21.15.190:0/3877258936 shutdown_connections 2024-04-21T10:24:03.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.815+0000 7f9750c44700 1 -- 172.21.15.190:0/3877258936 wait complete. 2024-04-21T10:24:03.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.816+0000 7f9750c44700 1 Processor -- start 2024-04-21T10:24:03.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.816+0000 7f9750c44700 1 -- start start 2024-04-21T10:24:03.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.816+0000 7f9750c44700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f974c100d90 0x7f974c19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:24:03.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.816+0000 7f9750c44700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c103d70 0x7f974c19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:24:03.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.816+0000 7f9750c44700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f974c104710 0x7f974c1a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:24:03.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.816+0000 7f9750c44700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f974c10e650 con 0x7f974c104710 2024-04-21T10:24:03.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.816+0000 7f9750c44700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f974c10e4d0 con 0x7f974c100d90 2024-04-21T10:24:03.818 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.816+0000 7f9750c44700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f974c10e7d0 con 0x7f974c103d70 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.816+0000 7f9749d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c103d70 0x7f974c19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.816+0000 7f9749d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c103d70 0x7f974c19b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:48294/0 (socket says 172.21.15.190:48294) 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.816+0000 7f9749d9b700 1 -- 172.21.15.190:0/1175597890 learned_addr learned my addr 172.21.15.190:0/1175597890 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f974a59c700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f974c100d90 0x7f974c19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f9749d9b700 1 -- 172.21.15.190:0/1175597890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f974c100d90 msgr2=0x7f974c19afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f9749d9b700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f974c100d90 0x7f974c19afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f974ad9d700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f974c104710 0x7f974c1a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f9749d9b700 1 -- 172.21.15.190:0/1175597890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f974c104710 msgr2=0x7f974c1a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f9749d9b700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f974c104710 0x7f974c1a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f9749d9b700 1 -- 172.21.15.190:0/1175597890 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c1a2bb0 con 0x7f974c103d70 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f974a59c700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f974c100d90 0x7f974c19afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f974ad9d700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f974c104710 0x7f974c1a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:24:03.819 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f9749d9b700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c103d70 0x7f974c19b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f973c0038a0 tx=0x7f973c0039f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:24:03.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f973b7fe700 1 -- 172.21.15.190:0/1175597890 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c027070 con 0x7f974c103d70 2024-04-21T10:24:03.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f973b7fe700 1 -- 172.21.15.190:0/1175597890 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f973c022470 con 0x7f974c103d70 2024-04-21T10:24:03.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f9750c44700 1 -- 172.21.15.190:0/1175597890 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974c1a2e40 con 0x7f974c103d70 2024-04-21T10:24:03.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f973b7fe700 1 -- 172.21.15.190:0/1175597890 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c00f6c0 con 0x7f974c103d70 2024-04-21T10:24:03.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.817+0000 7f9750c44700 1 -- 172.21.15.190:0/1175597890 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974c1a32b0 con 0x7f974c103d70 2024-04-21T10:24:03.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.818+0000 7f9750c44700 1 -- 172.21.15.190:0/1175597890 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f974c069730 con 0x7f974c103d70 2024-04-21T10:24:03.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.819+0000 7f973b7fe700 1 -- 172.21.15.190:0/1175597890 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f973c02a070 con 0x7f974c103d70 2024-04-21T10:24:03.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.820+0000 7f973b7fe700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f97340a3bb0 0x7f97340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:24:03.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.820+0000 7f973b7fe700 1 -- 172.21.15.190:0/1175597890 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f973c0cff40 con 0x7f974c103d70 2024-04-21T10:24:03.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.820+0000 7f974a59c700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f97340a3bb0 0x7f97340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:24:03.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.820+0000 7f974a59c700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f97340a3bb0 0x7f97340a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f974c06afb0 tx=0x7f9740008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:24:03.823 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:03.824+0000 7f973b7fe700 1 -- 172.21.15.190:0/1175597890 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f973c013070 con 0x7f974c103d70 2024-04-21T10:24:04.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:03 smithi190 ceph-mon[31423]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:24:04.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:03 smithi190 ceph-mon[31423]: Detected new or changed devices on smithi169 2024-04-21T10:24:04.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:03 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-21T10:24:04.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:03 smithi190 ceph-mon[31423]: osd.1 [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313] boot 2024-04-21T10:24:04.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:03 smithi190 ceph-mon[31423]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T10:24:04.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:04.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:04.028+0000 7f9750c44700 1 -- 172.21.15.190:0/1175597890 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f974c103280 con 0x7f97340a3bb0 2024-04-21T10:24:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:03 smithi026 ceph-mon[27965]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:24:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:03 smithi026 ceph-mon[27965]: Detected new or changed devices on smithi169 2024-04-21T10:24:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:03 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-21T10:24:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:03 smithi026 ceph-mon[27965]: osd.1 [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313] boot 2024-04-21T10:24:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:03 smithi026 ceph-mon[27965]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T10:24:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:03 smithi169 ceph-mon[31469]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T10:24:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:03 smithi169 ceph-mon[31469]: Detected new or changed devices on smithi169 2024-04-21T10:24:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:03 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-21T10:24:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:03 smithi169 ceph-mon[31469]: osd.1 [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313] boot 2024-04-21T10:24:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:03 smithi169 ceph-mon[31469]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T10:24:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:24:04.947 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:04 smithi190 ceph-mon[31423]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T10:24:04.947 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:04 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:24:04.947 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:04 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:24:04.947 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:04 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:05.086 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:04 smithi026 ceph-mon[27965]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T10:24:05.087 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:04 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:24:05.087 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:04 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:24:05.087 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:04 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:04 smithi169 ceph-mon[31469]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T10:24:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:04 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:24:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:04 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:24:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:04 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:05.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:05 smithi190 ceph-mon[31423]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:24:05.933 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:05 smithi190 ceph-mon[31423]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:06.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:05 smithi026 ceph-mon[27965]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:24:06.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:05 smithi026 ceph-mon[27965]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:06.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:05 smithi169 ceph-mon[31469]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:24:06.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:05 smithi169 ceph-mon[31469]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:07.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:06 smithi190 ceph-mon[31423]: from='client.? 172.21.15.190:0/22575742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7906664-6ec7-46e9-a500-50bac1bc7674"}]: dispatch 2024-04-21T10:24:07.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:06 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7906664-6ec7-46e9-a500-50bac1bc7674"}]: dispatch 2024-04-21T10:24:07.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:06 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7906664-6ec7-46e9-a500-50bac1bc7674"}]': finished 2024-04-21T10:24:07.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:06 smithi190 ceph-mon[31423]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T10:24:07.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:07.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:06 smithi026 ceph-mon[27965]: from='client.? 172.21.15.190:0/22575742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7906664-6ec7-46e9-a500-50bac1bc7674"}]: dispatch 2024-04-21T10:24:07.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:06 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7906664-6ec7-46e9-a500-50bac1bc7674"}]: dispatch 2024-04-21T10:24:07.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:06 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7906664-6ec7-46e9-a500-50bac1bc7674"}]': finished 2024-04-21T10:24:07.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:06 smithi026 ceph-mon[27965]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T10:24:07.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:07.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:06 smithi169 ceph-mon[31469]: from='client.? 172.21.15.190:0/22575742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7906664-6ec7-46e9-a500-50bac1bc7674"}]: dispatch 2024-04-21T10:24:07.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:06 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7906664-6ec7-46e9-a500-50bac1bc7674"}]: dispatch 2024-04-21T10:24:07.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:06 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7906664-6ec7-46e9-a500-50bac1bc7674"}]': finished 2024-04-21T10:24:07.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:06 smithi169 ceph-mon[31469]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T10:24:07.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:08.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:07 smithi190 ceph-mon[31423]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:08.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:07 smithi190 ceph-mon[31423]: from='client.? 172.21.15.190:0/2258585532' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:24:08.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:07 smithi026 ceph-mon[27965]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:08.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:07 smithi026 ceph-mon[27965]: from='client.? 172.21.15.190:0/2258585532' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:24:08.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:07 smithi169 ceph-mon[31469]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:08.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:07 smithi169 ceph-mon[31469]: from='client.? 172.21.15.190:0/2258585532' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:24:10.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:09 smithi190 ceph-mon[31423]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:10.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:09 smithi026 ceph-mon[27965]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:10.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:09 smithi169 ceph-mon[31469]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:11.969 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:11 smithi190 ceph-mon[31423]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:12.015 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:11 smithi169 ceph-mon[31469]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:12.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:11 smithi026 ceph-mon[27965]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:14.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:13 smithi190 ceph-mon[31423]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:14.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:13 smithi026 ceph-mon[27965]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:14.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:13 smithi169 ceph-mon[31469]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:15.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:14 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T10:24:15.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:14 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:15.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:14 smithi190 ceph-mon[31423]: Deploying daemon osd.2 on smithi190 2024-04-21T10:24:15.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T10:24:15.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:15.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:14 smithi026 ceph-mon[27965]: Deploying daemon osd.2 on smithi190 2024-04-21T10:24:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:14 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T10:24:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:14 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:14 smithi169 ceph-mon[31469]: Deploying daemon osd.2 on smithi190 2024-04-21T10:24:16.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:15 smithi190 ceph-mon[31423]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:16.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:15 smithi026 ceph-mon[27965]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:16.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:15 smithi169 ceph-mon[31469]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:18.179 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:17 smithi190 ceph-mon[31423]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:18.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:17 smithi026 ceph-mon[27965]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:18.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:17 smithi169 ceph-mon[31469]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:20.153 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:19 smithi190 ceph-mon[31423]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:20.153 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:19 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:20.154 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:19 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:20.154 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:19 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:24:20.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:19 smithi026 ceph-mon[27965]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:20.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:19 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:20.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:19 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:20.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:19 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:24:20.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:19 smithi169 ceph-mon[31469]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:20.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:19 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:20.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:19 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:20.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:19 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:24:22.043 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:21 smithi190 ceph-mon[31423]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:22.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:21 smithi026 ceph-mon[27965]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:22.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:21 smithi169 ceph-mon[31469]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:23.348 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.348+0000 7f973b7fe700 1 -- 172.21.15.190:0/1175597890 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f974c103280 con 0x7f97340a3bb0 2024-04-21T10:24:23.348 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 2 on host 'smithi190' 2024-04-21T10:24:23.350 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.350+0000 7f9750c44700 1 -- 172.21.15.190:0/1175597890 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f97340a3bb0 msgr2=0x7f97340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:23.350 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.350+0000 7f9750c44700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f97340a3bb0 0x7f97340a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f974c06afb0 tx=0x7f9740008040 comp rx=0 tx=0).stop 2024-04-21T10:24:23.350 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.351+0000 7f9750c44700 1 -- 172.21.15.190:0/1175597890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c103d70 msgr2=0x7f974c19b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:23.351 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.351+0000 7f9750c44700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c103d70 0x7f974c19b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f973c0038a0 tx=0x7f973c0039f0 comp rx=0 tx=0).stop 2024-04-21T10:24:23.351 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.351+0000 7f9750c44700 1 -- 172.21.15.190:0/1175597890 shutdown_connections 2024-04-21T10:24:23.351 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.351+0000 7f9750c44700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f97340a3bb0 0x7f97340a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:23.351 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.351+0000 7f9750c44700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f974c100d90 0x7f974c19afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:23.351 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.351+0000 7f9750c44700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f974c104710 0x7f974c1a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:23.351 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.351+0000 7f9750c44700 1 --2- 172.21.15.190:0/1175597890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c103d70 0x7f974c19b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:23.351 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.351+0000 7f9750c44700 1 -- 172.21.15.190:0/1175597890 >> 172.21.15.190:0/1175597890 conn(0x7f974c0fc6f0 msgr2=0x7f974c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:24:23.351 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.351+0000 7f9750c44700 1 -- 172.21.15.190:0/1175597890 shutdown_connections 2024-04-21T10:24:23.351 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:24:23.351+0000 7f9750c44700 1 -- 172.21.15.190:0/1175597890 wait complete. 2024-04-21T10:24:23.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:23 smithi169 ceph-mon[31469]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:23.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:23.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:23.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:24:23.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:23 smithi169 ceph-mon[31469]: from='osd.2 [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T10:24:23.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:23 smithi169 ceph-mon[31469]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T10:24:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:23 smithi190 ceph-mon[31423]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:24:23.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:23 smithi190 ceph-mon[31423]: from='osd.2 [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T10:24:23.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:23 smithi190 ceph-mon[31423]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T10:24:23.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:23 smithi026 ceph-mon[27965]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:23.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:24:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:23 smithi026 ceph-mon[27965]: from='osd.2 [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T10:24:23.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:23 smithi026 ceph-mon[27965]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T10:24:24.048 DEBUG:teuthology.orchestra.run.smithi190:osd.2> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.2.service 2024-04-21T10:24:24.050 INFO:tasks.cephadm:Deploying osd.3 on smithi026 with /dev/vg_nvme/lv_3... 2024-04-21T10:24:24.050 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T10:24:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:24:25.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:25 smithi026 ceph-mon[27965]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:25.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:25 smithi026 ceph-mon[27965]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T10:24:25.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:25 smithi026 ceph-mon[27965]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T10:24:25.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:25 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:25.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:25 smithi026 ceph-mon[27965]: from='osd.2 [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:24:25.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:25 smithi026 ceph-mon[27965]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:24:25.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:25 smithi169 ceph-mon[31469]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:25.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:25 smithi169 ceph-mon[31469]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T10:24:25.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:25 smithi169 ceph-mon[31469]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T10:24:25.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:25 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:25.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:25 smithi169 ceph-mon[31469]: from='osd.2 [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:24:25.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:25 smithi169 ceph-mon[31469]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:24:25.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:25 smithi190 ceph-mon[31423]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:25.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:25 smithi190 ceph-mon[31423]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T10:24:25.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:25 smithi190 ceph-mon[31423]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T10:24:25.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:25 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:25.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:25 smithi190 ceph-mon[31423]: from='osd.2 [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:24:25.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:25 smithi190 ceph-mon[31423]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:24:25.778 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:24:25 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2[36587]: 2024-04-21T10:24:25.361+0000 7f1db6d8f700 -1 osd.2 0 waiting for initial osdmap 2024-04-21T10:24:25.778 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:24:25 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2[36587]: 2024-04-21T10:24:25.375+0000 7f1db2b82700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T10:24:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T10:24:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T10:24:26.348 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-04-21T10:24:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-04-21T10:24:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00848147 s, 1.2 GB/s 2024-04-21T10:24:26.348 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 2024-04-21T10:24:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-04-21T10:24:26.626 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:26 smithi026 ceph-mon[27965]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-04-21T10:24:26.626 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:26 smithi026 ceph-mon[27965]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T10:24:26.626 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:26 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:26.626 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:26 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:26.693 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:26 smithi190 ceph-mon[31423]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-04-21T10:24:26.693 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:26 smithi190 ceph-mon[31423]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T10:24:26.693 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:26 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:26.693 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:26 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:26.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:26 smithi169 ceph-mon[31469]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-04-21T10:24:26.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:26 smithi169 ceph-mon[31469]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T10:24:26.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:26 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:26.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:26 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:26.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_3 2024-04-21T10:24:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:24:27.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: purged_snaps scrub starts 2024-04-21T10:24:27.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: purged_snaps scrub ok 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: osd.2 [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539] boot 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: Detected new or changed devices on smithi190 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi190 to 17088M 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:27.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:27.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:27.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: purged_snaps scrub starts 2024-04-21T10:24:27.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: purged_snaps scrub ok 2024-04-21T10:24:27.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:27.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: osd.2 [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539] boot 2024-04-21T10:24:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T10:24:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: Detected new or changed devices on smithi190 2024-04-21T10:24:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:24:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi190 to 17088M 2024-04-21T10:24:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:27.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: purged_snaps scrub starts 2024-04-21T10:24:27.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: purged_snaps scrub ok 2024-04-21T10:24:27.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-21T10:24:27.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:27.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: osd.2 [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539] boot 2024-04-21T10:24:27.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T10:24:27.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:24:27.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: Detected new or changed devices on smithi190 2024-04-21T10:24:27.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:27.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:27.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:24:27.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi190 to 17088M 2024-04-21T10:24:27.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:27.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:27.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:27.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:28.497 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:28 smithi026 ceph-mon[27965]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T10:24:28.497 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:28 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T10:24:28.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:28 smithi169 ceph-mon[31469]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T10:24:28.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:28 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T10:24:28.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.737+0000 7f6eade5c700 1 -- 172.21.15.26:0/4235777238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8104640 msgr2=0x7f6ea810aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:28.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.737+0000 7f6eade5c700 1 --2- 172.21.15.26:0/4235777238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8104640 0x7f6ea810aef0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000b0d0 tx=0x7f6ea000b3e0 comp rx=0 tx=0).stop 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.737+0000 7f6eade5c700 1 -- 172.21.15.26:0/4235777238 shutdown_connections 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.737+0000 7f6eade5c700 1 --2- 172.21.15.26:0/4235777238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ea8102aa0 0x7f6ea8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.737+0000 7f6eade5c700 1 --2- 172.21.15.26:0/4235777238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8104640 0x7f6ea810aef0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.737+0000 7f6eade5c700 1 --2- 172.21.15.26:0/4235777238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea8103ca0 0x7f6ea8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.737+0000 7f6eade5c700 1 -- 172.21.15.26:0/4235777238 >> 172.21.15.26:0/4235777238 conn(0x7f6ea80fe040 msgr2=0x7f6ea8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.737+0000 7f6eade5c700 1 -- 172.21.15.26:0/4235777238 shutdown_connections 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.737+0000 7f6eade5c700 1 -- 172.21.15.26:0/4235777238 wait complete. 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.738+0000 7f6eade5c700 1 Processor -- start 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.738+0000 7f6eade5c700 1 -- start start 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.738+0000 7f6eade5c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea8102aa0 0x7f6ea8198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.738+0000 7f6eade5c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8103ca0 0x7f6ea81990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.738+0000 7f6eade5c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ea8104640 0x7f6ea81a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:24:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.738+0000 7f6eade5c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea810e590 con 0x7f6ea8103ca0 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.738+0000 7f6eade5c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea810e410 con 0x7f6ea8104640 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.738+0000 7f6eade5c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea810e710 con 0x7f6ea8102aa0 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.738+0000 7f6ea6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8103ca0 0x7f6ea81990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8103ca0 0x7f6ea81990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53088/0 (socket says 172.21.15.26:53088) 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea8102aa0 0x7f6ea8198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea6ffd700 1 -- 172.21.15.26:0/3431923856 learned_addr learned my addr 172.21.15.26:0/3431923856 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea7fff700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ea8104640 0x7f6ea81a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea6ffd700 1 -- 172.21.15.26:0/3431923856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea8102aa0 msgr2=0x7f6ea8198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea6ffd700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea8102aa0 0x7f6ea8198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea6ffd700 1 -- 172.21.15.26:0/3431923856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ea8104640 msgr2=0x7f6ea81a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea6ffd700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ea8104640 0x7f6ea81a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea6ffd700 1 -- 172.21.15.26:0/3431923856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea81a0800 con 0x7f6ea8103ca0 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea6ffd700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8103ca0 0x7f6ea81990e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00c9a0 tx=0x7f6e9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:24:28.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea4ff9700 1 -- 172.21.15.26:0/3431923856 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c004a30 con 0x7f6ea8103ca0 2024-04-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea4ff9700 1 -- 172.21.15.26:0/3431923856 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e9c0054d0 con 0x7f6ea8103ca0 2024-04-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6ea4ff9700 1 -- 172.21.15.26:0/3431923856 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c007bd0 con 0x7f6ea8103ca0 2024-04-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6eade5c700 1 -- 172.21.15.26:0/3431923856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea81a0af0 con 0x7f6ea8103ca0 2024-04-21T10:24:28.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.739+0000 7f6eade5c700 1 -- 172.21.15.26:0/3431923856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea81a0fe0 con 0x7f6ea8103ca0 2024-04-21T10:24:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.740+0000 7f6eade5c700 1 -- 172.21.15.26:0/3431923856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea8069730 con 0x7f6ea8103ca0 2024-04-21T10:24:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.741+0000 7f6ea4ff9700 1 -- 172.21.15.26:0/3431923856 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f6e9c004bd0 con 0x7f6ea8103ca0 2024-04-21T10:24:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.742+0000 7f6ea4ff9700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6e900a3bb0 0x7f6e900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:24:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.742+0000 7f6ea77fe700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6e900a3bb0 0x7f6e900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:24:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.742+0000 7f6ea4ff9700 1 -- 172.21.15.26:0/3431923856 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f6e9c0cebe0 con 0x7f6ea8103ca0 2024-04-21T10:24:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.743+0000 7f6ea77fe700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6e900a3bb0 0x7f6e900a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8103b00 tx=0x7f6e98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:24:28.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.747+0000 7f6ea4ff9700 1 -- 172.21.15.26:0/3431923856 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e9c09cb10 con 0x7f6ea8103ca0 2024-04-21T10:24:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:28 smithi190 ceph-mon[31423]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T10:24:28.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:28 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T10:24:28.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:28.964+0000 7f6eade5c700 1 -- 172.21.15.26:0/3431923856 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6ea8062e80 con 0x7f6e900a3bb0 2024-04-21T10:24:29.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 ceph-mon[27965]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:29.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T10:24:29.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 ceph-mon[27965]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T10:24:29.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T10:24:29.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:24:29.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:24:29.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:29.629 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44248]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T10:24:29.629 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44248]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T10:24:29.629 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44248]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T10:24:29.629 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44248]: pam_unix(sudo:session): session closed for user root 2024-04-21T10:24:29.629 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44263]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T10:24:29.629 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44263]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T10:24:29.629 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44263]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T10:24:29.629 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44263]: pam_unix(sudo:session): session closed for user root 2024-04-21T10:24:29.636 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40194]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T10:24:29.636 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40194]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T10:24:29.636 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40194]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T10:24:29.637 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40194]: pam_unix(sudo:session): session closed for user root 2024-04-21T10:24:29.637 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 ceph-mon[31423]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:29.637 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T10:24:29.637 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 ceph-mon[31423]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T10:24:29.637 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T10:24:29.637 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:24:29.637 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:24:29.637 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:29.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 ceph-mon[31469]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:29.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T10:24:29.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 ceph-mon[31469]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T10:24:29.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T10:24:29.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:24:29.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:24:29.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:29.699 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39756]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T10:24:29.699 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39756]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T10:24:29.699 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39756]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T10:24:29.699 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39756]: pam_unix(sudo:session): session closed for user root 2024-04-21T10:24:29.699 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39759]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T10:24:29.699 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39759]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T10:24:29.699 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39759]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T10:24:29.699 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39759]: pam_unix(sudo:session): session closed for user root 2024-04-21T10:24:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44268]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T10:24:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44268]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T10:24:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44268]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T10:24:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44268]: pam_unix(sudo:session): session closed for user root 2024-04-21T10:24:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44271]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T10:24:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44271]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T10:24:29.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44271]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T10:24:29.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:29 smithi026 sudo[44271]: pam_unix(sudo:session): session closed for user root 2024-04-21T10:24:30.027 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40197]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T10:24:30.027 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40197]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T10:24:30.027 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40197]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T10:24:30.027 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40197]: pam_unix(sudo:session): session closed for user root 2024-04-21T10:24:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40200]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T10:24:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40200]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T10:24:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40200]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T10:24:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40200]: pam_unix(sudo:session): session closed for user root 2024-04-21T10:24:30.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40203]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T10:24:30.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40203]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T10:24:30.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40203]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T10:24:30.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:29 smithi190 sudo[40203]: pam_unix(sudo:session): session closed for user root 2024-04-21T10:24:30.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39762]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T10:24:30.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39762]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T10:24:30.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39762]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T10:24:30.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39762]: pam_unix(sudo:session): session closed for user root 2024-04-21T10:24:30.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39765]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T10:24:30.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39765]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T10:24:30.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39765]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T10:24:30.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:29 smithi169 sudo[39765]: pam_unix(sudo:session): session closed for user root 2024-04-21T10:24:30.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:24:30.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T10:24:30.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T10:24:30.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:30.509 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:24:30.510 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:24:30.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:30 smithi026 ceph-mon[27965]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T10:24:30.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:30.673+0000 7f6ea4ff9700 1 -- 172.21.15.26:0/3431923856 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6e9c098a80 con 0x7f6ea8103ca0 2024-04-21T10:24:30.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:24:30.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:24:30.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T10:24:30.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T10:24:30.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:24:30.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:24:30.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:24:30.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:30 smithi169 ceph-mon[31469]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T10:24:30.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:24:30.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T10:24:30.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T10:24:30.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:24:30.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:24:30.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:24:30.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:30 smithi190 ceph-mon[31423]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T10:24:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:31 smithi026 ceph-mon[27965]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:31 smithi026 ceph-mon[27965]: osdmap e22: 3 total, 3 up, 3 in 2024-04-21T10:24:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:31 smithi026 ceph-mon[27965]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T10:24:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:31 smithi169 ceph-mon[31469]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:31 smithi169 ceph-mon[31469]: osdmap e22: 3 total, 3 up, 3 in 2024-04-21T10:24:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:31 smithi169 ceph-mon[31469]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T10:24:31.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:31 smithi190 ceph-mon[31423]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:31.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:31 smithi190 ceph-mon[31423]: osdmap e22: 3 total, 3 up, 3 in 2024-04-21T10:24:31.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:31 smithi190 ceph-mon[31423]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T10:24:32.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:32 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/723203901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62499ade-dc25-4d67-8e9f-5486c2298565"}]: dispatch 2024-04-21T10:24:32.386 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:32 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/723203901' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62499ade-dc25-4d67-8e9f-5486c2298565"}]': finished 2024-04-21T10:24:32.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:32 smithi026 ceph-mon[27965]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T10:24:32.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:32 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:32.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:32 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1218031662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:24:32.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:32 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/723203901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62499ade-dc25-4d67-8e9f-5486c2298565"}]: dispatch 2024-04-21T10:24:32.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:32 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/723203901' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62499ade-dc25-4d67-8e9f-5486c2298565"}]': finished 2024-04-21T10:24:32.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:32 smithi169 ceph-mon[31469]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T10:24:32.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:32 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:32.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:32 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1218031662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:24:32.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:32 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/723203901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62499ade-dc25-4d67-8e9f-5486c2298565"}]: dispatch 2024-04-21T10:24:32.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:32 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/723203901' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62499ade-dc25-4d67-8e9f-5486c2298565"}]': finished 2024-04-21T10:24:32.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:32 smithi190 ceph-mon[31423]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T10:24:32.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:32 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:32.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:32 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1218031662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:24:33.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:33 smithi026 ceph-mon[27965]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:33.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:33 smithi169 ceph-mon[31469]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:34.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:33 smithi190 ceph-mon[31423]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:35.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:35 smithi026 ceph-mon[27965]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:35.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:35 smithi169 ceph-mon[31469]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:36.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:35 smithi190 ceph-mon[31423]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:37.824 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:37 smithi026 ceph-mon[27965]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:37.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:37 smithi169 ceph-mon[31469]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:38.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:37 smithi190 ceph-mon[31423]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:39.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:39 smithi026 ceph-mon[27965]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:39.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:39 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T10:24:39.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:39 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:39.812 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:39 smithi026 ceph-mon[27965]: Deploying daemon osd.3 on smithi026 2024-04-21T10:24:39.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:39 smithi169 ceph-mon[31469]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:39.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:39 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T10:24:39.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:39 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:39.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:39 smithi169 ceph-mon[31469]: Deploying daemon osd.3 on smithi026 2024-04-21T10:24:40.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:39 smithi190 ceph-mon[31423]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:40.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:39 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T10:24:40.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:39 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:40.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:39 smithi190 ceph-mon[31423]: Deploying daemon osd.3 on smithi026 2024-04-21T10:24:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:41 smithi026 ceph-mon[27965]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:41.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:41 smithi169 ceph-mon[31469]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:42.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:41 smithi190 ceph-mon[31423]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:43.929 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:43 smithi026 ceph-mon[27965]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:43.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:43 smithi169 ceph-mon[31469]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:44.013 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:43 smithi190 ceph-mon[31423]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:45.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:45 smithi026 ceph-mon[27965]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:45.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:45 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:45.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:45 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:45.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:45 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:24:45.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:45 smithi190 ceph-mon[31423]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:45.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:45 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:45.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:45 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:45.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:45 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:24:45.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:45 smithi169 ceph-mon[31469]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:45.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:45 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:45.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:45 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:45.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:45 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:24:47.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:47 smithi190 ceph-mon[31423]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:47.579 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:47 smithi026 ceph-mon[27965]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.666+0000 7f6ea4ff9700 1 -- 172.21.15.26:0/3431923856 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ea8062e80 con 0x7f6e900a3bb0 2024-04-21T10:24:47.666 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 3 on host 'smithi026' 2024-04-21T10:24:47.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.668+0000 7f6eade5c700 1 -- 172.21.15.26:0/3431923856 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6e900a3bb0 msgr2=0x7f6e900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:47.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.668+0000 7f6eade5c700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6e900a3bb0 0x7f6e900a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8103b00 tx=0x7f6e98009260 comp rx=0 tx=0).stop 2024-04-21T10:24:47.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.668+0000 7f6eade5c700 1 -- 172.21.15.26:0/3431923856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8103ca0 msgr2=0x7f6ea81990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.668+0000 7f6eade5c700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8103ca0 0x7f6ea81990e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00c9a0 tx=0x7f6e9c00ccb0 comp rx=0 tx=0).stop 2024-04-21T10:24:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.668+0000 7f6eade5c700 1 -- 172.21.15.26:0/3431923856 shutdown_connections 2024-04-21T10:24:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.668+0000 7f6eade5c700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6e900a3bb0 0x7f6e900a6070 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.668+0000 7f6eade5c700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ea8104640 0x7f6ea81a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.668+0000 7f6eade5c700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ea8102aa0 0x7f6ea8198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.668+0000 7f6eade5c700 1 --2- 172.21.15.26:0/3431923856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea8103ca0 0x7f6ea81990e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.668+0000 7f6eade5c700 1 -- 172.21.15.26:0/3431923856 >> 172.21.15.26:0/3431923856 conn(0x7f6ea80fe040 msgr2=0x7f6ea8107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:24:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.669+0000 7f6eade5c700 1 -- 172.21.15.26:0/3431923856 shutdown_connections 2024-04-21T10:24:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:24:47.669+0000 7f6eade5c700 1 -- 172.21.15.26:0/3431923856 wait complete. 2024-04-21T10:24:47.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:47 smithi169 ceph-mon[31469]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:48.311 DEBUG:teuthology.orchestra.run.smithi026:osd.3> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.3.service 2024-04-21T10:24:48.313 INFO:tasks.cephadm:Deploying osd.4 on smithi169 with /dev/vg_nvme/lv_3... 2024-04-21T10:24:48.313 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T10:24:48.561 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T10:24:48.924 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:48 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:48.924 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:48 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:48.924 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:48 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:48.924 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:48 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:48.924 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:48 smithi169 ceph-mon[31469]: from='osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T10:24:48.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:48 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:48.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:48 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:48.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:48 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:48.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:48 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:48.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:48 smithi026 ceph-mon[27965]: from='osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T10:24:49.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:48 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:49.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:48 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:49.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:48 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:49.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:48 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:49.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:48 smithi190 ceph-mon[31423]: from='osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T10:24:49.910 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:49 smithi169 ceph-mon[31469]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:49.910 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:49 smithi169 ceph-mon[31469]: from='osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T10:24:49.910 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:49 smithi169 ceph-mon[31469]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T10:24:49.910 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:49 smithi169 ceph-mon[31469]: from='osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-04-21T10:24:49.910 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:49.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:49 smithi026 ceph-mon[27965]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:49.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:49 smithi026 ceph-mon[27965]: from='osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T10:24:49.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:49 smithi026 ceph-mon[27965]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T10:24:49.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:49 smithi026 ceph-mon[27965]: from='osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-04-21T10:24:49.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:49.936 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:24:49 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3[46410]: 2024-04-21T10:24:49.727+0000 7fa87483f700 -1 osd.3 0 waiting for initial osdmap 2024-04-21T10:24:49.936 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:24:49 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3[46410]: 2024-04-21T10:24:49.732+0000 7fa870632700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T10:24:50.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:49 smithi190 ceph-mon[31423]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:50.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:49 smithi190 ceph-mon[31423]: from='osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T10:24:50.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:49 smithi190 ceph-mon[31423]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T10:24:50.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:49 smithi190 ceph-mon[31423]: from='osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-04-21T10:24:50.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:50.478 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T10:24:50.478 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T10:24:50.478 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-21T10:24:50.478 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-21T10:24:50.478 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115057 s, 911 MB/s 2024-04-21T10:24:50.479 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 2024-04-21T10:24:50.479 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-21T10:24:51.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:50 smithi190 ceph-mon[31423]: from='osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-04-21T10:24:51.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:50 smithi190 ceph-mon[31423]: osdmap e25: 4 total, 3 up, 4 in 2024-04-21T10:24:51.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:51.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:50 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:51.053 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:50 smithi169 ceph-mon[31469]: from='osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-04-21T10:24:51.053 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:50 smithi169 ceph-mon[31469]: osdmap e25: 4 total, 3 up, 4 in 2024-04-21T10:24:51.053 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:50 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:51.053 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:50 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:51.080 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_3 2024-04-21T10:24:51.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:50 smithi026 ceph-mon[27965]: from='osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-04-21T10:24:51.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:50 smithi026 ceph-mon[27965]: osdmap e25: 4 total, 3 up, 4 in 2024-04-21T10:24:51.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:51.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:50 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:51.334 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T10:24:52.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: purged_snaps scrub starts 2024-04-21T10:24:52.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: purged_snaps scrub ok 2024-04-21T10:24:52.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:52.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:52.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169] boot 2024-04-21T10:24:52.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T10:24:52.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:52.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: Detected new or changed devices on smithi026 2024-04-21T10:24:52.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:24:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:24:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:24:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:52.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.181 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: purged_snaps scrub starts 2024-04-21T10:24:52.181 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: purged_snaps scrub ok 2024-04-21T10:24:52.182 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:52.182 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:52.182 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169] boot 2024-04-21T10:24:52.182 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T10:24:52.182 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:52.182 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: Detected new or changed devices on smithi026 2024-04-21T10:24:52.182 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.182 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.182 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:24:52.182 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:24:52.182 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.182 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:52.183 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:52.183 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.183 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:24:52.183 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:52.183 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:52.183 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: purged_snaps scrub starts 2024-04-21T10:24:52.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: purged_snaps scrub ok 2024-04-21T10:24:52.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-21T10:24:52.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: osd.3 [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169] boot 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: Detected new or changed devices on smithi026 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:24:52.187 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:52.187 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:24:52.187 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:24:52.842 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:52 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi026 to 8546M 2024-04-21T10:24:52.842 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:52 smithi169 ceph-mon[31469]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T10:24:52.950 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.948+0000 7fbfded36700 1 -- 172.21.15.169:0/3256136902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd8103cc0 msgr2=0x7fbfd8104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:52.950 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.948+0000 7fbfded36700 1 --2- 172.21.15.169:0/3256136902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd8103cc0 0x7fbfd8104120 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40099c0 tx=0x7fbfc4009cd0 comp rx=0 tx=0).stop 2024-04-21T10:24:52.950 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.948+0000 7fbfded36700 1 -- 172.21.15.169:0/3256136902 shutdown_connections 2024-04-21T10:24:52.950 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.948+0000 7fbfded36700 1 --2- 172.21.15.169:0/3256136902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfd8104660 0x7fbfd810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:52.950 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.948+0000 7fbfded36700 1 --2- 172.21.15.169:0/3256136902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfd8102ac0 0x7fbfd8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:52.950 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.948+0000 7fbfded36700 1 --2- 172.21.15.169:0/3256136902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd8103cc0 0x7fbfd8104120 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:52.950 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.948+0000 7fbfded36700 1 -- 172.21.15.169:0/3256136902 >> 172.21.15.169:0/3256136902 conn(0x7fbfd80fe060 msgr2=0x7fbfd81004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:24:52.950 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfded36700 1 -- 172.21.15.169:0/3256136902 shutdown_connections 2024-04-21T10:24:52.950 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfded36700 1 -- 172.21.15.169:0/3256136902 wait complete. 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfded36700 1 Processor -- start 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfded36700 1 -- start start 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfded36700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfd8102ac0 0x7fbfd8198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfded36700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfd8103cc0 0x7fbfd8199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfded36700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd8104660 0x7fbfd81a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfded36700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810e560 con 0x7fbfd8103cc0 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfded36700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810e3e0 con 0x7fbfd8104660 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfded36700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810e6e0 con 0x7fbfd8102ac0 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfdd2d3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd8104660 0x7fbfd81a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfdd2d3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd8104660 0x7fbfd81a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55442/0 (socket says 172.21.15.169:55442) 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfdd2d3700 1 -- 172.21.15.169:0/3083006312 learned_addr learned my addr 172.21.15.169:0/3083006312 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.949+0000 7fbfdcad2700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfd8102ac0 0x7fbfd8198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.950+0000 7fbfd7fff700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfd8103cc0 0x7fbfd8199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:24:52.951 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.950+0000 7fbfdd2d3700 1 -- 172.21.15.169:0/3083006312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfd8102ac0 msgr2=0x7fbfd8198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:52.952 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.950+0000 7fbfdd2d3700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfd8102ac0 0x7fbfd8198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:52.952 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.950+0000 7fbfdd2d3700 1 -- 172.21.15.169:0/3083006312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfd8103cc0 msgr2=0x7fbfd8199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:24:52.952 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.950+0000 7fbfdd2d3700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfd8103cc0 0x7fbfd8199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:24:52.952 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.950+0000 7fbfdd2d3700 1 -- 172.21.15.169:0/3083006312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd81a07b0 con 0x7fbfd8104660 2024-04-21T10:24:52.952 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.950+0000 7fbfdd2d3700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd8104660 0x7fbfd81a0110 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000ada0 tx=0x7fbfd000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:24:52.952 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.950+0000 7fbfd5ffb700 1 -- 172.21.15.169:0/3083006312 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0017020 con 0x7fbfd8104660 2024-04-21T10:24:52.952 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.950+0000 7fbfded36700 1 -- 172.21.15.169:0/3083006312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd81a0aa0 con 0x7fbfd8104660 2024-04-21T10:24:52.952 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.950+0000 7fbfd5ffb700 1 -- 172.21.15.169:0/3083006312 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfd000cd80 con 0x7fbfd8104660 2024-04-21T10:24:52.952 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.950+0000 7fbfd5ffb700 1 -- 172.21.15.169:0/3083006312 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0005280 con 0x7fbfd8104660 2024-04-21T10:24:52.952 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.950+0000 7fbfded36700 1 -- 172.21.15.169:0/3083006312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd81a1030 con 0x7fbfd8104660 2024-04-21T10:24:52.953 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.951+0000 7fbfded36700 1 -- 172.21.15.169:0/3083006312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd8069730 con 0x7fbfd8104660 2024-04-21T10:24:52.953 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.952+0000 7fbfd5ffb700 1 -- 172.21.15.169:0/3083006312 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbfd000c8d0 con 0x7fbfd8104660 2024-04-21T10:24:52.953 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.953+0000 7fbfd5ffb700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbfc80a3c40 0x7fbfc80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:24:52.953 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.953+0000 7fbfd5ffb700 1 -- 172.21.15.169:0/3083006312 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7fbfd00ce8a0 con 0x7fbfd8104660 2024-04-21T10:24:52.953 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.953+0000 7fbfdcad2700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbfc80a3c40 0x7fbfc80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:24:52.953 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.953+0000 7fbfdcad2700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbfc80a3c40 0x7fbfc80a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8103b20 tx=0x7fbfcc00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:24:52.957 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:52.958+0000 7fbfd5ffb700 1 -- 172.21.15.169:0/3083006312 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbfd0021020 con 0x7fbfd8104660 2024-04-21T10:24:53.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:52 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi026 to 8546M 2024-04-21T10:24:53.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:52 smithi190 ceph-mon[31423]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T10:24:53.162 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:24:53.162+0000 7fbfded36700 1 -- 172.21.15.169:0/3083006312 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbfd8062e80 con 0x7fbfc80a3c40 2024-04-21T10:24:53.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:52 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi026 to 8546M 2024-04-21T10:24:53.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:52 smithi026 ceph-mon[27965]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T10:24:53.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:53 smithi190 ceph-mon[31423]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:24:53.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:53 smithi190 ceph-mon[31423]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T10:24:53.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:53 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:24:53.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:53 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:24:53.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:53 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:54.088 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:53 smithi169 ceph-mon[31469]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:24:54.088 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:53 smithi169 ceph-mon[31469]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T10:24:54.088 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:53 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:24:54.088 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:53 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:24:54.088 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:53 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:54.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:53 smithi026 ceph-mon[27965]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:24:54.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:53 smithi026 ceph-mon[27965]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T10:24:54.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:53 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:24:54.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:53 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:24:54.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:53 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:24:54.993 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:54 smithi169 ceph-mon[31469]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:24:54.993 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:54 smithi169 ceph-mon[31469]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T10:24:55.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:54 smithi190 ceph-mon[31423]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:24:55.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:54 smithi190 ceph-mon[31423]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T10:24:55.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:54 smithi026 ceph-mon[27965]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:24:55.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:54 smithi026 ceph-mon[27965]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T10:24:56.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:55 smithi190 ceph-mon[31423]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:24:56.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:55 smithi190 ceph-mon[31423]: osdmap e30: 4 total, 4 up, 4 in 2024-04-21T10:24:56.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:55 smithi026 ceph-mon[27965]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:24:56.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:55 smithi026 ceph-mon[27965]: osdmap e30: 4 total, 4 up, 4 in 2024-04-21T10:24:56.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:55 smithi169 ceph-mon[31469]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:24:56.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:55 smithi169 ceph-mon[31469]: osdmap e30: 4 total, 4 up, 4 in 2024-04-21T10:24:57.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:56 smithi190 ceph-mon[31423]: from='client.? 172.21.15.169:0/2566522665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b50fcf-cfcd-42a2-b246-eb9d736aa437"}]: dispatch 2024-04-21T10:24:57.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:56 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b50fcf-cfcd-42a2-b246-eb9d736aa437"}]: dispatch 2024-04-21T10:24:57.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:56 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1b50fcf-cfcd-42a2-b246-eb9d736aa437"}]': finished 2024-04-21T10:24:57.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:56 smithi190 ceph-mon[31423]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T10:24:57.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:24:57.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:56 smithi190 ceph-mon[31423]: from='client.? 172.21.15.169:0/48250433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:24:57.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:56 smithi026 ceph-mon[27965]: from='client.? 172.21.15.169:0/2566522665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b50fcf-cfcd-42a2-b246-eb9d736aa437"}]: dispatch 2024-04-21T10:24:57.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:56 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b50fcf-cfcd-42a2-b246-eb9d736aa437"}]: dispatch 2024-04-21T10:24:57.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:56 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1b50fcf-cfcd-42a2-b246-eb9d736aa437"}]': finished 2024-04-21T10:24:57.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:56 smithi026 ceph-mon[27965]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T10:24:57.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:24:57.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:56 smithi026 ceph-mon[27965]: from='client.? 172.21.15.169:0/48250433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:24:57.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:56 smithi169 ceph-mon[31469]: from='client.? 172.21.15.169:0/2566522665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b50fcf-cfcd-42a2-b246-eb9d736aa437"}]: dispatch 2024-04-21T10:24:57.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:56 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b50fcf-cfcd-42a2-b246-eb9d736aa437"}]: dispatch 2024-04-21T10:24:57.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:56 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1b50fcf-cfcd-42a2-b246-eb9d736aa437"}]': finished 2024-04-21T10:24:57.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:56 smithi169 ceph-mon[31469]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T10:24:57.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:24:57.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:56 smithi169 ceph-mon[31469]: from='client.? 172.21.15.169:0/48250433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:24:58.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:57 smithi026 ceph-mon[27965]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:24:58.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:57 smithi169 ceph-mon[31469]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:24:58.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:57 smithi190 ceph-mon[31423]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:25:00.106 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:24:59 smithi169 ceph-mon[31469]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-21T10:25:00.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:24:59 smithi026 ceph-mon[27965]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-21T10:25:00.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:24:59 smithi190 ceph-mon[31423]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-21T10:25:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:01 smithi026 ceph-mon[27965]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:02.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:01 smithi169 ceph-mon[31469]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:02.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:01 smithi190 ceph-mon[31423]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:03.986 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:03 smithi169 ceph-mon[31469]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:03.986 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T10:25:03.986 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:03 smithi026 ceph-mon[27965]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T10:25:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:04.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:03 smithi190 ceph-mon[31423]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:04.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T10:25:04.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:05.063 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:04 smithi169 ceph-mon[31469]: Deploying daemon osd.4 on smithi169 2024-04-21T10:25:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:04 smithi026 ceph-mon[27965]: Deploying daemon osd.4 on smithi169 2024-04-21T10:25:05.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:04 smithi190 ceph-mon[31423]: Deploying daemon osd.4 on smithi169 2024-04-21T10:25:06.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:05 smithi190 ceph-mon[31423]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:06.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:05 smithi026 ceph-mon[27965]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:06.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:05 smithi169 ceph-mon[31469]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:07.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:07 smithi169 ceph-mon[31469]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:08.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:07 smithi026 ceph-mon[27965]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:08.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:07 smithi190 ceph-mon[31423]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:09.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:09 smithi190 ceph-mon[31423]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:09.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:09.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:09.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:09.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:09 smithi026 ceph-mon[27965]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:09.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:09.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:09.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:09.940 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:09 smithi169 ceph-mon[31469]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:09.940 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:09.940 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:09.940 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:11.624 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:11 smithi169 ceph-mon[31469]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:11.625 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:11 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:11.625 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:11 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:11.625 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:11 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:11.625 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:11 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:25:11.625 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:11 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:11.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:11 smithi190 ceph-mon[31423]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:11.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:11 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:11.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:11 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:11.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:11 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:11.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:11 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:25:11.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:11 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:11.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:11 smithi026 ceph-mon[27965]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:11.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:11 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:11.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:11 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:11.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:11 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:11.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:11 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:25:11.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:11 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:11.979 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.980+0000 7fbfd5ffb700 1 -- 172.21.15.169:0/3083006312 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbfd8062e80 con 0x7fbfc80a3c40 2024-04-21T10:25:11.979 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 4 on host 'smithi169' 2024-04-21T10:25:11.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.982+0000 7fbfded36700 1 -- 172.21.15.169:0/3083006312 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbfc80a3c40 msgr2=0x7fbfc80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:25:11.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.982+0000 7fbfded36700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbfc80a3c40 0x7fbfc80a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8103b20 tx=0x7fbfcc00a3b0 comp rx=0 tx=0).stop 2024-04-21T10:25:11.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.982+0000 7fbfded36700 1 -- 172.21.15.169:0/3083006312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd8104660 msgr2=0x7fbfd81a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:25:11.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.982+0000 7fbfded36700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd8104660 0x7fbfd81a0110 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000ada0 tx=0x7fbfd000c330 comp rx=0 tx=0).stop 2024-04-21T10:25:11.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.982+0000 7fbfded36700 1 -- 172.21.15.169:0/3083006312 shutdown_connections 2024-04-21T10:25:11.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.982+0000 7fbfded36700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbfc80a3c40 0x7fbfc80a6100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:11.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.982+0000 7fbfded36700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd8104660 0x7fbfd81a0110 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:11.982 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.982+0000 7fbfded36700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfd8102ac0 0x7fbfd8198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:11.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.982+0000 7fbfded36700 1 --2- 172.21.15.169:0/3083006312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfd8103cc0 0x7fbfd8199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:11.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.982+0000 7fbfded36700 1 -- 172.21.15.169:0/3083006312 >> 172.21.15.169:0/3083006312 conn(0x7fbfd80fe060 msgr2=0x7fbfd8106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:25:11.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.982+0000 7fbfded36700 1 -- 172.21.15.169:0/3083006312 shutdown_connections 2024-04-21T10:25:11.983 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:25:11.982+0000 7fbfded36700 1 -- 172.21.15.169:0/3083006312 wait complete. 2024-04-21T10:25:12.789 DEBUG:teuthology.orchestra.run.smithi169:osd.4> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.4.service 2024-04-21T10:25:12.791 INFO:tasks.cephadm:Deploying osd.5 on smithi190 with /dev/vg_nvme/lv_3... 2024-04-21T10:25:12.792 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T10:25:13.036 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:25:13.038 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:13.038 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:13.038 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:13.038 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:12 smithi169 ceph-mon[31469]: from='osd.4 [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T10:25:13.038 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:12 smithi169 ceph-mon[31469]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T10:25:13.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:13.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:13.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:13.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:12 smithi190 ceph-mon[31423]: from='osd.4 [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T10:25:13.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:12 smithi190 ceph-mon[31423]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T10:25:13.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:13.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:13.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:13.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:12 smithi026 ceph-mon[27965]: from='osd.4 [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T10:25:13.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:12 smithi026 ceph-mon[27965]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T10:25:14.221 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:13 smithi169 ceph-mon[31469]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:14.221 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:13 smithi169 ceph-mon[31469]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T10:25:14.221 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:13 smithi169 ceph-mon[31469]: from='osd.4 [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:25:14.221 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:13 smithi169 ceph-mon[31469]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T10:25:14.221 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:13 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:14.221 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:13 smithi169 ceph-mon[31469]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:25:14.241 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:13 smithi190 ceph-mon[31423]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:14.242 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:13 smithi190 ceph-mon[31423]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T10:25:14.242 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:13 smithi190 ceph-mon[31423]: from='osd.4 [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:25:14.242 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:13 smithi190 ceph-mon[31423]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T10:25:14.242 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:13 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:14.242 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:13 smithi190 ceph-mon[31423]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:25:14.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:13 smithi026 ceph-mon[27965]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:14.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:13 smithi026 ceph-mon[27965]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T10:25:14.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:13 smithi026 ceph-mon[27965]: from='osd.4 [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:25:14.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:13 smithi026 ceph-mon[27965]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T10:25:14.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:13 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:14.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:13 smithi026 ceph-mon[27965]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:25:14.923 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T10:25:14.924 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T10:25:14.924 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-04-21T10:25:14.924 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-04-21T10:25:14.924 INFO:teuthology.orchestra.run.smithi190.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114484 s, 916 MB/s 2024-04-21T10:25:14.924 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-04-21T10:25:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:14 smithi169 ceph-mon[31469]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-21T10:25:15.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:14 smithi169 ceph-mon[31469]: osdmap e33: 5 total, 4 up, 5 in 2024-04-21T10:25:15.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:14 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:15.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:14 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:15.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:14 smithi169 ceph-mon[31469]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:15.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:14 smithi169 ceph-mon[31469]: from='osd.4 ' entity='osd.4' 2024-04-21T10:25:15.199 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:25:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4[42562]: 2024-04-21T10:25:14.912+0000 7fbbb2181700 -1 osd.4 0 waiting for initial osdmap 2024-04-21T10:25:15.199 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:25:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4[42562]: 2024-04-21T10:25:14.995+0000 7fbba9f6c700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T10:25:15.260 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:14 smithi190 ceph-mon[31423]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-21T10:25:15.260 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:14 smithi190 ceph-mon[31423]: osdmap e33: 5 total, 4 up, 5 in 2024-04-21T10:25:15.260 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:14 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:15.260 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:14 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:15.260 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:14 smithi190 ceph-mon[31423]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:15.260 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:14 smithi190 ceph-mon[31423]: from='osd.4 ' entity='osd.4' 2024-04-21T10:25:15.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:14 smithi026 ceph-mon[27965]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-21T10:25:15.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:14 smithi026 ceph-mon[27965]: osdmap e33: 5 total, 4 up, 5 in 2024-04-21T10:25:15.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:15.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:14 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:15.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:14 smithi026 ceph-mon[27965]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:15.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:14 smithi026 ceph-mon[27965]: from='osd.4 ' entity='osd.4' 2024-04-21T10:25:15.624 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_3 2024-04-21T10:25:15.867 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:25:16.231 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:15 smithi190 ceph-mon[31423]: purged_snaps scrub starts 2024-04-21T10:25:16.231 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:15 smithi190 ceph-mon[31423]: purged_snaps scrub ok 2024-04-21T10:25:16.231 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:16.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:15 smithi026 ceph-mon[27965]: purged_snaps scrub starts 2024-04-21T10:25:16.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:15 smithi026 ceph-mon[27965]: purged_snaps scrub ok 2024-04-21T10:25:16.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:16.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:15 smithi169 ceph-mon[31469]: purged_snaps scrub starts 2024-04-21T10:25:16.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:15 smithi169 ceph-mon[31469]: purged_snaps scrub ok 2024-04-21T10:25:16.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:17.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: osd.4 [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642] boot 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: Detected new or changed devices on smithi169 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:17.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:16 smithi026 ceph-mon[27965]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:17.206 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: osd.4 [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642] boot 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: Detected new or changed devices on smithi169 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:17.207 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:16 smithi190 ceph-mon[31423]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:17.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.408+0000 7fb2cfe24700 1 -- 172.21.15.190:0/4075482299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8103c90 msgr2=0x7fb2c8106120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:25:17.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.408+0000 7fb2cfe24700 1 --2- 172.21.15.190:0/4075482299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8103c90 0x7fb2c8106120 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009990 tx=0x7fb2bc009ca0 comp rx=0 tx=0).stop 2024-04-21T10:25:17.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.408+0000 7fb2cfe24700 1 -- 172.21.15.190:0/4075482299 shutdown_connections 2024-04-21T10:25:17.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.408+0000 7fb2cfe24700 1 --2- 172.21.15.190:0/4075482299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2c8106660 0x7fb2c8106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:17.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.408+0000 7fb2cfe24700 1 --2- 172.21.15.190:0/4075482299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c8101320 0x7fb2c8103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.408+0000 7fb2cfe24700 1 --2- 172.21.15.190:0/4075482299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8103c90 0x7fb2c8106120 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.408+0000 7fb2cfe24700 1 -- 172.21.15.190:0/4075482299 >> 172.21.15.190:0/4075482299 conn(0x7fb2c80fcf40 msgr2=0x7fb2c80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.408+0000 7fb2cfe24700 1 -- 172.21.15.190:0/4075482299 shutdown_connections 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.408+0000 7fb2cfe24700 1 -- 172.21.15.190:0/4075482299 wait complete. 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cfe24700 1 Processor -- start 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cfe24700 1 -- start start 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cfe24700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2c8101320 0x7fb2c819f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cfe24700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8103c90 0x7fb2c819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cfe24700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c8106660 0x7fb2c81a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cfe24700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8114b10 con 0x7fb2c8106660 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cfe24700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8114990 con 0x7fb2c8101320 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cfe24700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8114c90 con 0x7fb2c8103c90 2024-04-21T10:25:17.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cd3bf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8103c90 0x7fb2c819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cd3bf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8103c90 0x7fb2c819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:52726/0 (socket says 172.21.15.190:52726) 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cd3bf700 1 -- 172.21.15.190:0/696624805 learned_addr learned my addr 172.21.15.190:0/696624805 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2ce3c1700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c8106660 0x7fb2c81a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cd3bf700 1 -- 172.21.15.190:0/696624805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2c8101320 msgr2=0x7fb2c819f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cdbc0700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2c8101320 0x7fb2c819f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cd3bf700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2c8101320 0x7fb2c819f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cd3bf700 1 -- 172.21.15.190:0/696624805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c8106660 msgr2=0x7fb2c81a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cd3bf700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c8106660 0x7fb2c81a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cd3bf700 1 -- 172.21.15.190:0/696624805 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c81a6e40 con 0x7fb2c8103c90 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2ce3c1700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c8106660 0x7fb2c81a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cdbc0700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2c8101320 0x7fb2c819f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.409+0000 7fb2cd3bf700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8103c90 0x7fb2c819f720 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0059d0 tx=0x7fb2bc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.410+0000 7fb2c6ffd700 1 -- 172.21.15.190:0/696624805 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc027070 con 0x7fb2c8103c90 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.410+0000 7fb2c6ffd700 1 -- 172.21.15.190:0/696624805 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2bc004080 con 0x7fb2c8103c90 2024-04-21T10:25:17.412 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.410+0000 7fb2cfe24700 1 -- 172.21.15.190:0/696624805 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c81a70d0 con 0x7fb2c8103c90 2024-04-21T10:25:17.413 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.410+0000 7fb2c6ffd700 1 -- 172.21.15.190:0/696624805 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc022cf0 con 0x7fb2c8103c90 2024-04-21T10:25:17.413 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.410+0000 7fb2cfe24700 1 -- 172.21.15.190:0/696624805 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c81a7660 con 0x7fb2c8103c90 2024-04-21T10:25:17.413 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.411+0000 7fb2cfe24700 1 -- 172.21.15.190:0/696624805 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c8069730 con 0x7fb2c8103c90 2024-04-21T10:25:17.413 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.412+0000 7fb2c6ffd700 1 -- 172.21.15.190:0/696624805 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb2bc02a070 con 0x7fb2c8103c90 2024-04-21T10:25:17.413 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.412+0000 7fb2c6ffd700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb2b00a3c40 0x7fb2b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:25:17.414 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.412+0000 7fb2c6ffd700 1 -- 172.21.15.190:0/696624805 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fb2bc0d0370 con 0x7fb2c8103c90 2024-04-21T10:25:17.414 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.412+0000 7fb2cdbc0700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb2b00a3c40 0x7fb2b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:25:17.414 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.413+0000 7fb2cdbc0700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb2b00a3c40 0x7fb2b00a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80094f0 tx=0x7fb2b8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:25:17.416 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.417+0000 7fb2c6ffd700 1 -- 172.21.15.190:0/696624805 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2bc05eff0 con 0x7fb2c8103c90 2024-04-21T10:25:17.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: osd.4 [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642] boot 2024-04-21T10:25:17.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T10:25:17.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:25:17.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: Detected new or changed devices on smithi169 2024-04-21T10:25:17.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:17.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:17.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:25:17.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:25:17.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-21T10:25:17.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:17.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:17.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:25:17.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:17.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:16 smithi169 ceph-mon[31469]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-21T10:25:17.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:17.623+0000 7fb2cfe24700 1 -- 172.21.15.190:0/696624805 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb2c8062e80 con 0x7fb2b00a3c40 2024-04-21T10:25:18.431 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:18 smithi190 ceph-mon[31423]: osdmap e35: 5 total, 5 up, 5 in 2024-04-21T10:25:18.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:18 smithi190 ceph-mon[31423]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:25:18.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:18 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:25:18.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:18 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:25:18.545 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:18 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:18.546 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:18 smithi169 ceph-mon[31469]: osdmap e35: 5 total, 5 up, 5 in 2024-04-21T10:25:18.546 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:18 smithi169 ceph-mon[31469]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:25:18.546 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:18 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:25:18.547 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:18 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:25:18.547 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:18 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:18.547 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:18 smithi026 ceph-mon[27965]: osdmap e35: 5 total, 5 up, 5 in 2024-04-21T10:25:18.547 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:18 smithi026 ceph-mon[27965]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:25:18.547 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:18 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:25:18.547 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:18 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:25:18.547 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:18 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:19.160 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:19 smithi190 ceph-mon[31423]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:19.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:19 smithi026 ceph-mon[27965]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:19.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:19 smithi169 ceph-mon[31469]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:21.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:21 smithi190 ceph-mon[31423]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:21.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:21 smithi190 ceph-mon[31423]: from='client.? 172.21.15.190:0/2002691456' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33e19659-9fe5-4fe5-92e4-167c782735b5"}]: dispatch 2024-04-21T10:25:21.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:21 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33e19659-9fe5-4fe5-92e4-167c782735b5"}]: dispatch 2024-04-21T10:25:21.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:21 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33e19659-9fe5-4fe5-92e4-167c782735b5"}]': finished 2024-04-21T10:25:21.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:21 smithi190 ceph-mon[31423]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T10:25:21.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:21 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:21.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:21 smithi190 ceph-mon[31423]: from='client.? 172.21.15.190:0/394379323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:25:21.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:21 smithi026 ceph-mon[27965]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:21.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:21 smithi026 ceph-mon[27965]: from='client.? 172.21.15.190:0/2002691456' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33e19659-9fe5-4fe5-92e4-167c782735b5"}]: dispatch 2024-04-21T10:25:21.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:21 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33e19659-9fe5-4fe5-92e4-167c782735b5"}]: dispatch 2024-04-21T10:25:21.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:21 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33e19659-9fe5-4fe5-92e4-167c782735b5"}]': finished 2024-04-21T10:25:21.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:21 smithi026 ceph-mon[27965]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T10:25:21.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:21 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:21.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:21 smithi026 ceph-mon[27965]: from='client.? 172.21.15.190:0/394379323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:25:21.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:21 smithi169 ceph-mon[31469]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:21.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:21 smithi169 ceph-mon[31469]: from='client.? 172.21.15.190:0/2002691456' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33e19659-9fe5-4fe5-92e4-167c782735b5"}]: dispatch 2024-04-21T10:25:21.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:21 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33e19659-9fe5-4fe5-92e4-167c782735b5"}]: dispatch 2024-04-21T10:25:21.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:21 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33e19659-9fe5-4fe5-92e4-167c782735b5"}]': finished 2024-04-21T10:25:21.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:21 smithi169 ceph-mon[31469]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T10:25:21.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:21 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:21.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:21 smithi169 ceph-mon[31469]: from='client.? 172.21.15.190:0/394379323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:25:23.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:23 smithi190 ceph-mon[31423]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:23 smithi026 ceph-mon[27965]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:23.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:23 smithi169 ceph-mon[31469]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:25.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:25 smithi169 ceph-mon[31469]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:25.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:25 smithi190 ceph-mon[31423]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:25.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:25 smithi026 ceph-mon[27965]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:27.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:27 smithi190 ceph-mon[31423]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:27 smithi026 ceph-mon[27965]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:27.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:27 smithi169 ceph-mon[31469]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:28.460 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:28 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T10:25:28.460 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:28 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:28.460 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:28 smithi190 ceph-mon[31423]: Deploying daemon osd.5 on smithi190 2024-04-21T10:25:28.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:28 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T10:25:28.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:28 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:28.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:28 smithi026 ceph-mon[27965]: Deploying daemon osd.5 on smithi190 2024-04-21T10:25:28.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:28 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T10:25:28.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:28 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:28.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:28 smithi169 ceph-mon[31469]: Deploying daemon osd.5 on smithi190 2024-04-21T10:25:29.321 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:29 smithi190 ceph-mon[31423]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:29.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:29 smithi026 ceph-mon[27965]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:29.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:29 smithi169 ceph-mon[31469]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:31.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:31 smithi190 ceph-mon[31423]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:31 smithi026 ceph-mon[27965]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:31 smithi169 ceph-mon[31469]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:33.247 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:33 smithi190 ceph-mon[31423]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:33 smithi026 ceph-mon[27965]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:33 smithi169 ceph-mon[31469]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:34.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:34 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:34.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:34 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:34.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:34 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:34.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:34.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:34.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:34.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:34 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:34.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:34 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:34.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:34 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:35.586 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:35 smithi026 ceph-mon[27965]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:35.628 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:35 smithi190 ceph-mon[31423]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:35 smithi169 ceph-mon[31469]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:36.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.992+0000 7fb2c6ffd700 1 -- 172.21.15.190:0/696624805 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb2c8062e80 con 0x7fb2b00a3c40 2024-04-21T10:25:36.992 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 5 on host 'smithi190' 2024-04-21T10:25:36.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.994+0000 7fb2cfe24700 1 -- 172.21.15.190:0/696624805 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb2b00a3c40 msgr2=0x7fb2b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:25:36.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.994+0000 7fb2cfe24700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb2b00a3c40 0x7fb2b00a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80094f0 tx=0x7fb2b8009040 comp rx=0 tx=0).stop 2024-04-21T10:25:36.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.994+0000 7fb2cfe24700 1 -- 172.21.15.190:0/696624805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8103c90 msgr2=0x7fb2c819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:25:36.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.994+0000 7fb2cfe24700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8103c90 0x7fb2c819f720 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0059d0 tx=0x7fb2bc009f90 comp rx=0 tx=0).stop 2024-04-21T10:25:36.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.994+0000 7fb2cfe24700 1 -- 172.21.15.190:0/696624805 shutdown_connections 2024-04-21T10:25:36.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.994+0000 7fb2cfe24700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb2b00a3c40 0x7fb2b00a6100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:36.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.994+0000 7fb2cfe24700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2c8101320 0x7fb2c819f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:36.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.994+0000 7fb2cfe24700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c8106660 0x7fb2c81a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:36.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.994+0000 7fb2cfe24700 1 --2- 172.21.15.190:0/696624805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2c8103c90 0x7fb2c819f720 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:36.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.994+0000 7fb2cfe24700 1 -- 172.21.15.190:0/696624805 >> 172.21.15.190:0/696624805 conn(0x7fb2c80fcf40 msgr2=0x7fb2c8101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:25:36.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.994+0000 7fb2cfe24700 1 -- 172.21.15.190:0/696624805 shutdown_connections 2024-04-21T10:25:36.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:25:36.994+0000 7fb2cfe24700 1 -- 172.21.15.190:0/696624805 wait complete. 2024-04-21T10:25:37.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:37.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:25:37.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:37 smithi190 ceph-mon[31423]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:37.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:37.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:37.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:25:37.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:37 smithi026 ceph-mon[27965]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:37.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:37.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:37.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:25:37.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:37 smithi169 ceph-mon[31469]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:37.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:37.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:37.559 DEBUG:teuthology.orchestra.run.smithi190:osd.5> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.5.service 2024-04-21T10:25:37.562 INFO:tasks.cephadm:Deploying osd.6 on smithi026 with /dev/vg_nvme/lv_2... 2024-04-21T10:25:37.562 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T10:25:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:25:38.188 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:38 smithi026 ceph-mon[27965]: from='osd.5 [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T10:25:38.188 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:38 smithi026 ceph-mon[27965]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T10:25:38.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:38 smithi190 ceph-mon[31423]: from='osd.5 [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T10:25:38.235 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:38 smithi190 ceph-mon[31423]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T10:25:38.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:38 smithi169 ceph-mon[31469]: from='osd.5 [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T10:25:38.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:38 smithi169 ceph-mon[31469]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T10:25:39.361 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:39 smithi026 ceph-mon[27965]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T10:25:39.361 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:39 smithi026 ceph-mon[27965]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T10:25:39.361 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:39 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:39.361 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:39 smithi026 ceph-mon[27965]: from='osd.5 [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:25:39.361 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:39 smithi026 ceph-mon[27965]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:25:39.361 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:39 smithi026 ceph-mon[27965]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:39 smithi169 ceph-mon[31469]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T10:25:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:39 smithi169 ceph-mon[31469]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T10:25:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:39 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:39 smithi169 ceph-mon[31469]: from='osd.5 [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:25:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:39 smithi169 ceph-mon[31469]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:25:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:39 smithi169 ceph-mon[31469]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:39.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:39 smithi190 ceph-mon[31423]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T10:25:39.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:39 smithi190 ceph-mon[31423]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T10:25:39.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:39 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:39.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:39 smithi190 ceph-mon[31423]: from='osd.5 [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:25:39.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:39 smithi190 ceph-mon[31423]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:25:39.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:39 smithi190 ceph-mon[31423]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T10:25:39.943 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T10:25:39.943 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-04-21T10:25:39.943 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-04-21T10:25:39.943 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0084744 s, 1.2 GB/s 2024-04-21T10:25:39.943 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-04-21T10:25:40.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:40 smithi190 ceph-mon[31423]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-04-21T10:25:40.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:40 smithi190 ceph-mon[31423]: osdmap e38: 6 total, 5 up, 6 in 2024-04-21T10:25:40.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:40.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:40.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:40 smithi190 ceph-mon[31423]: from='osd.5 ' entity='osd.5' 2024-04-21T10:25:40.115 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:25:39 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:25:39.969+0000 7fe902a67700 -1 osd.5 0 waiting for initial osdmap 2024-04-21T10:25:40.115 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:25:40 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:25:40.114+0000 7fe8fa852700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T10:25:40.248 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:40 smithi026 ceph-mon[27965]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-04-21T10:25:40.248 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:40 smithi026 ceph-mon[27965]: osdmap e38: 6 total, 5 up, 6 in 2024-04-21T10:25:40.248 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:40.248 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:40.248 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:40 smithi026 ceph-mon[27965]: from='osd.5 ' entity='osd.5' 2024-04-21T10:25:40.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:40 smithi169 ceph-mon[31469]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-04-21T10:25:40.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:40 smithi169 ceph-mon[31469]: osdmap e38: 6 total, 5 up, 6 in 2024-04-21T10:25:40.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:40.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:40.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:40 smithi169 ceph-mon[31469]: from='osd.5 ' entity='osd.5' 2024-04-21T10:25:40.532 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_2 2024-04-21T10:25:40.781 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:25:41.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: purged_snaps scrub starts 2024-04-21T10:25:41.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: purged_snaps scrub ok 2024-04-21T10:25:41.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:41.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: osdmap e39: 6 total, 5 up, 6 in 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: Detected new or changed devices on smithi190 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi190 to 8544M 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:41.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:41 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:41.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: purged_snaps scrub starts 2024-04-21T10:25:41.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: purged_snaps scrub ok 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: osdmap e39: 6 total, 5 up, 6 in 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: Detected new or changed devices on smithi190 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi190 to 8544M 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:25:41.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:41.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:41 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:41.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: purged_snaps scrub starts 2024-04-21T10:25:41.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: purged_snaps scrub ok 2024-04-21T10:25:41.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:41.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: osdmap e39: 6 total, 5 up, 6 in 2024-04-21T10:25:41.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:41.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-21T10:25:41.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: Detected new or changed devices on smithi190 2024-04-21T10:25:41.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:41.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:41.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:25:41.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:25:41.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi190 to 8544M 2024-04-21T10:25:41.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:41.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:41.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:25:41.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:41.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:41 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:42.106 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:42 smithi026 ceph-mon[27965]: osd.5 [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914] boot 2024-04-21T10:25:42.106 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:42 smithi026 ceph-mon[27965]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T10:25:42.106 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:42 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.357+0000 7f99d03e3700 1 -- 172.21.15.26:0/2405564454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99c8105e80 msgr2=0x7f99c8108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:25:42.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.357+0000 7f99d03e3700 1 --2- 172.21.15.26:0/2405564454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99c8105e80 0x7f99c8108270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009990 tx=0x7f99b8009ca0 comp rx=0 tx=0).stop 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.358+0000 7f99d03e3700 1 -- 172.21.15.26:0/2405564454 shutdown_connections 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.358+0000 7f99d03e3700 1 --2- 172.21.15.26:0/2405564454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c81087b0 0x7f99c810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.358+0000 7f99d03e3700 1 --2- 172.21.15.26:0/2405564454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c8103550 0x7f99c8105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.358+0000 7f99d03e3700 1 --2- 172.21.15.26:0/2405564454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99c8105e80 0x7f99c8108270 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.358+0000 7f99d03e3700 1 -- 172.21.15.26:0/2405564454 >> 172.21.15.26:0/2405564454 conn(0x7f99c80fcf60 msgr2=0x7f99c80ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.358+0000 7f99d03e3700 1 -- 172.21.15.26:0/2405564454 shutdown_connections 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.358+0000 7f99d03e3700 1 -- 172.21.15.26:0/2405564454 wait complete. 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.358+0000 7f99d03e3700 1 Processor -- start 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.358+0000 7f99d03e3700 1 -- start start 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.358+0000 7f99d03e3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c8103550 0x7f99c8102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99d03e3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99c8105e80 0x7f99c8101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99d03e3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c81087b0 0x7f99c8101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99d03e3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f99c810e550 con 0x7f99c8105e80 2024-04-21T10:25:42.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99d03e3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f99c810e3d0 con 0x7f99c8103550 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99d03e3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f99c810e6d0 con 0x7f99c81087b0 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99cd97e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99c8105e80 0x7f99c8101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99cd97e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99c8105e80 0x7f99c8101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39934/0 (socket says 172.21.15.26:39934) 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99cd97e700 1 -- 172.21.15.26:0/2225548835 learned_addr learned my addr 172.21.15.26:0/2225548835 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99ce17f700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c8103550 0x7f99c8102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99ce980700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c81087b0 0x7f99c8101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99cd97e700 1 -- 172.21.15.26:0/2225548835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c81087b0 msgr2=0x7f99c8101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99cd97e700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c81087b0 0x7f99c8101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99cd97e700 1 -- 172.21.15.26:0/2225548835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c8103550 msgr2=0x7f99c8102eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99cd97e700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c8103550 0x7f99c8102eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99cd97e700 1 -- 172.21.15.26:0/2225548835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c81022f0 con 0x7f99c8105e80 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.359+0000 7f99cd97e700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99c8105e80 0x7f99c8101500 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f99b8006010 tx=0x7f99b80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.360+0000 7f99bf7fe700 1 -- 172.21.15.26:0/2225548835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8027070 con 0x7f99c8105e80 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.360+0000 7f99bf7fe700 1 -- 172.21.15.26:0/2225548835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b80040c0 con 0x7f99c8105e80 2024-04-21T10:25:42.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.360+0000 7f99d03e3700 1 -- 172.21.15.26:0/2225548835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c81a2ca0 con 0x7f99c8105e80 2024-04-21T10:25:42.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.360+0000 7f99bf7fe700 1 -- 172.21.15.26:0/2225548835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8022d40 con 0x7f99c8105e80 2024-04-21T10:25:42.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.360+0000 7f99d03e3700 1 -- 172.21.15.26:0/2225548835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c81a30f0 con 0x7f99c8105e80 2024-04-21T10:25:42.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.361+0000 7f99d03e3700 1 -- 172.21.15.26:0/2225548835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c8069730 con 0x7f99c8105e80 2024-04-21T10:25:42.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.362+0000 7f99bf7fe700 1 -- 172.21.15.26:0/2225548835 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f99b802a070 con 0x7f99c8105e80 2024-04-21T10:25:42.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.363+0000 7f99bf7fe700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f99b40a3c40 0x7f99b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:25:42.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.363+0000 7f99bf7fe700 1 -- 172.21.15.26:0/2225548835 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f99b80d0410 con 0x7f99c8105e80 2024-04-21T10:25:42.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.363+0000 7f99ce17f700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f99b40a3c40 0x7f99b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:25:42.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.363+0000 7f99ce17f700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f99b40a3c40 0x7f99b40a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f99c400ac50 tx=0x7f99c400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:25:42.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.368+0000 7f99bf7fe700 1 -- 172.21.15.26:0/2225548835 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99b8013070 con 0x7f99c8105e80 2024-04-21T10:25:42.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:42 smithi169 ceph-mon[31469]: osd.5 [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914] boot 2024-04-21T10:25:42.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:42 smithi169 ceph-mon[31469]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T10:25:42.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:42 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:42.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:42 smithi190 ceph-mon[31423]: osd.5 [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914] boot 2024-04-21T10:25:42.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:42 smithi190 ceph-mon[31423]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T10:25:42.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:42 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:25:42.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:25:42.585+0000 7f99d03e3700 1 -- 172.21.15.26:0/2225548835 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f99c8062e80 con 0x7f99b40a3c40 2024-04-21T10:25:43.333 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:43 smithi026 ceph-mon[27965]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T10:25:43.333 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:43 smithi026 ceph-mon[27965]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:43.333 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:43 smithi026 ceph-mon[27965]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:25:43.333 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:25:43.333 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:25:43.333 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:43.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:43 smithi169 ceph-mon[31469]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T10:25:43.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:43 smithi169 ceph-mon[31469]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:43.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:43 smithi169 ceph-mon[31469]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:25:43.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:25:43.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:25:43.350 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:43.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:43 smithi190 ceph-mon[31423]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T10:25:43.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:43 smithi190 ceph-mon[31423]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:43.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:43 smithi190 ceph-mon[31423]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:25:43.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:25:43.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:25:43.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:44.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:44 smithi026 ceph-mon[27965]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T10:25:44.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:44 smithi026 ceph-mon[27965]: osdmap e43: 6 total, 6 up, 6 in 2024-04-21T10:25:44.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:44 smithi169 ceph-mon[31469]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T10:25:44.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:44 smithi169 ceph-mon[31469]: osdmap e43: 6 total, 6 up, 6 in 2024-04-21T10:25:44.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:44 smithi190 ceph-mon[31423]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T10:25:44.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:44 smithi190 ceph-mon[31423]: osdmap e43: 6 total, 6 up, 6 in 2024-04-21T10:25:46.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:45 smithi190 ceph-mon[31423]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:25:46.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:45 smithi190 ceph-mon[31423]: osdmap e44: 6 total, 6 up, 6 in 2024-04-21T10:25:46.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:45 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1015994854' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cf2d5b-6cea-4f55-9b15-2feeea193919"}]: dispatch 2024-04-21T10:25:46.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:45 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1015994854' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20cf2d5b-6cea-4f55-9b15-2feeea193919"}]': finished 2024-04-21T10:25:46.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:45 smithi190 ceph-mon[31423]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T10:25:46.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:45 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:25:46.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:45 smithi026 ceph-mon[27965]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:25:46.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:45 smithi026 ceph-mon[27965]: osdmap e44: 6 total, 6 up, 6 in 2024-04-21T10:25:46.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:45 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1015994854' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cf2d5b-6cea-4f55-9b15-2feeea193919"}]: dispatch 2024-04-21T10:25:46.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:45 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1015994854' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20cf2d5b-6cea-4f55-9b15-2feeea193919"}]': finished 2024-04-21T10:25:46.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:45 smithi026 ceph-mon[27965]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T10:25:46.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:45 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:25:46.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:45 smithi169 ceph-mon[31469]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:25:46.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:45 smithi169 ceph-mon[31469]: osdmap e44: 6 total, 6 up, 6 in 2024-04-21T10:25:46.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:45 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1015994854' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cf2d5b-6cea-4f55-9b15-2feeea193919"}]: dispatch 2024-04-21T10:25:46.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:45 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1015994854' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20cf2d5b-6cea-4f55-9b15-2feeea193919"}]': finished 2024-04-21T10:25:46.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:45 smithi169 ceph-mon[31469]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T10:25:46.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:45 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:25:47.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:46 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3299721376' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:25:47.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:46 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3299721376' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:25:47.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:46 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3299721376' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:25:48.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:47 smithi190 ceph-mon[31423]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:25:48.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:47 smithi026 ceph-mon[27965]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:25:48.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:47 smithi169 ceph-mon[31469]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:25:49.856 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:49 smithi026 ceph-mon[27965]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-21T10:25:50.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:49 smithi190 ceph-mon[31423]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-21T10:25:50.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:49 smithi169 ceph-mon[31469]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-21T10:25:51.827 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:51 smithi026 ceph-mon[27965]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:52.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:51 smithi190 ceph-mon[31423]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:52.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:51 smithi169 ceph-mon[31469]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:54.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:53 smithi190 ceph-mon[31423]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:54.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:53 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T10:25:54.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:53 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:54.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:53 smithi190 ceph-mon[31423]: Deploying daemon osd.6 on smithi026 2024-04-21T10:25:54.062 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:53 smithi169 ceph-mon[31469]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:54.063 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:53 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T10:25:54.063 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:53 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:54.063 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:53 smithi169 ceph-mon[31469]: Deploying daemon osd.6 on smithi026 2024-04-21T10:25:54.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:53 smithi026 ceph-mon[27965]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:54.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:53 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T10:25:54.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:53 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:25:54.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:53 smithi026 ceph-mon[27965]: Deploying daemon osd.6 on smithi026 2024-04-21T10:25:55.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:55 smithi026 ceph-mon[27965]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:56.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:55 smithi190 ceph-mon[31423]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:56.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:55 smithi169 ceph-mon[31469]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:57.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:57 smithi026 ceph-mon[27965]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:58.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:57 smithi190 ceph-mon[31423]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:58.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:57 smithi169 ceph-mon[31469]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:59 smithi169 ceph-mon[31469]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:59 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:59 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:25:59 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:59.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:59 smithi190 ceph-mon[31423]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:59.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:59.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:59.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:25:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:25:59.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:59 smithi026 ceph-mon[27965]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:25:59.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:59.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:25:59.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:25:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:01.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:01 smithi026 ceph-mon[27965]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:26:01.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:01.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:01.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:01.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:01.524 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:01.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:01 smithi169 ceph-mon[31469]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:26:01.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:01.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:01.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:01.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:01.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:01 smithi190 ceph-mon[31423]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:26:01.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:01.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:01.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:01.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:01.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.241+0000 7f99bf7fe700 1 -- 172.21.15.26:0/2225548835 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f99c8062e80 con 0x7f99b40a3c40 2024-04-21T10:26:02.241 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 6 on host 'smithi026' 2024-04-21T10:26:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.243+0000 7f99d03e3700 1 -- 172.21.15.26:0/2225548835 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f99b40a3c40 msgr2=0x7f99b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.243+0000 7f99d03e3700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f99b40a3c40 0x7f99b40a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f99c400ac50 tx=0x7f99c400a380 comp rx=0 tx=0).stop 2024-04-21T10:26:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.243+0000 7f99d03e3700 1 -- 172.21.15.26:0/2225548835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99c8105e80 msgr2=0x7f99c8101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.243+0000 7f99d03e3700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99c8105e80 0x7f99c8101500 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f99b8006010 tx=0x7f99b80057c0 comp rx=0 tx=0).stop 2024-04-21T10:26:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.243+0000 7f99d03e3700 1 -- 172.21.15.26:0/2225548835 shutdown_connections 2024-04-21T10:26:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.243+0000 7f99d03e3700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f99b40a3c40 0x7f99b40a6100 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.243+0000 7f99d03e3700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c8103550 0x7f99c8102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.243+0000 7f99d03e3700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c81087b0 0x7f99c8101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.243+0000 7f99d03e3700 1 --2- 172.21.15.26:0/2225548835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99c8105e80 0x7f99c8101500 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.243+0000 7f99d03e3700 1 -- 172.21.15.26:0/2225548835 >> 172.21.15.26:0/2225548835 conn(0x7f99c80fcf60 msgr2=0x7f99c8103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:26:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.243+0000 7f99d03e3700 1 -- 172.21.15.26:0/2225548835 shutdown_connections 2024-04-21T10:26:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:02.243+0000 7f99d03e3700 1 -- 172.21.15.26:0/2225548835 wait complete. 2024-04-21T10:26:02.926 DEBUG:teuthology.orchestra.run.smithi026:osd.6> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.6.service 2024-04-21T10:26:02.929 INFO:tasks.cephadm:Deploying osd.7 on smithi169 with /dev/vg_nvme/lv_2... 2024-04-21T10:26:02.929 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T10:26:03.174 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T10:26:03.469 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:03 smithi169 ceph-mon[31469]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:26:03.469 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:03.469 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:03.469 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:03.469 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:03 smithi169 ceph-mon[31469]: from='osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T10:26:03.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:03 smithi190 ceph-mon[31423]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:26:03.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:03.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:03.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:03.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:03 smithi190 ceph-mon[31423]: from='osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T10:26:03.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:03 smithi026 ceph-mon[27965]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:26:03.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:03.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:03.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:03.536 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:03 smithi026 ceph-mon[27965]: from='osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T10:26:04.398 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:04 smithi169 ceph-mon[31469]: from='osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T10:26:04.398 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:04 smithi169 ceph-mon[31469]: osdmap e46: 7 total, 6 up, 7 in 2024-04-21T10:26:04.398 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:04 smithi169 ceph-mon[31469]: from='osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-04-21T10:26:04.398 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:04 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:04.484 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:04 smithi026 ceph-mon[27965]: from='osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T10:26:04.484 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:04 smithi026 ceph-mon[27965]: osdmap e46: 7 total, 6 up, 7 in 2024-04-21T10:26:04.485 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:04 smithi026 ceph-mon[27965]: from='osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-04-21T10:26:04.485 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:04 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:04.485 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:26:04 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:26:04.230+0000 7fd09545b700 -1 osd.6 0 waiting for initial osdmap 2024-04-21T10:26:04.485 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:26:04 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:26:04.240+0000 7fd09124e700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T10:26:04.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:04 smithi190 ceph-mon[31423]: from='osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T10:26:04.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:04 smithi190 ceph-mon[31423]: osdmap e46: 7 total, 6 up, 7 in 2024-04-21T10:26:04.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:04 smithi190 ceph-mon[31423]: from='osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-04-21T10:26:04.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:04 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:05.331 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T10:26:05.331 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T10:26:05.331 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-21T10:26:05.331 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-21T10:26:05.331 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132406 s, 792 MB/s 2024-04-21T10:26:05.331 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-21T10:26:05.484 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:05 smithi169 ceph-mon[31469]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:26:05.484 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:05 smithi169 ceph-mon[31469]: from='osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-04-21T10:26:05.484 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:05 smithi169 ceph-mon[31469]: osdmap e47: 7 total, 6 up, 7 in 2024-04-21T10:26:05.484 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:05 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:05.484 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:05 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:05.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:05 smithi190 ceph-mon[31423]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:26:05.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:05 smithi190 ceph-mon[31423]: from='osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-04-21T10:26:05.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:05 smithi190 ceph-mon[31423]: osdmap e47: 7 total, 6 up, 7 in 2024-04-21T10:26:05.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:05 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:05.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:05 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:05.620 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:05 smithi026 ceph-mon[27965]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:26:05.620 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:05 smithi026 ceph-mon[27965]: from='osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-04-21T10:26:05.620 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:05 smithi026 ceph-mon[27965]: osdmap e47: 7 total, 6 up, 7 in 2024-04-21T10:26:05.620 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:05.620 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:06.024 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_2 2024-04-21T10:26:06.266 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T10:26:06.390 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: purged_snaps scrub starts 2024-04-21T10:26:06.390 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: purged_snaps scrub ok 2024-04-21T10:26:06.390 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:06.390 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337] boot 2024-04-21T10:26:06.390 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: osdmap e48: 7 total, 7 up, 7 in 2024-04-21T10:26:06.390 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:06.390 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:06.390 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:06.390 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:06.390 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:06.390 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:06.391 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:06.391 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:06.391 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:06.391 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:06 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:06.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: purged_snaps scrub starts 2024-04-21T10:26:06.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: purged_snaps scrub ok 2024-04-21T10:26:06.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:06.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337] boot 2024-04-21T10:26:06.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: osdmap e48: 7 total, 7 up, 7 in 2024-04-21T10:26:06.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:06.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:06.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:06.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:06.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:06.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:06.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:06.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:06.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:06.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:06 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:06.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: purged_snaps scrub starts 2024-04-21T10:26:06.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: purged_snaps scrub ok 2024-04-21T10:26:06.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:06.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: osd.6 [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337] boot 2024-04-21T10:26:06.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: osdmap e48: 7 total, 7 up, 7 in 2024-04-21T10:26:06.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:26:06.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:06.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:06.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:06.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:06.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:06.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:06.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:06.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:06.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:06 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:07 smithi169 ceph-mon[31469]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:26:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:07 smithi169 ceph-mon[31469]: Detected new or changed devices on smithi026 2024-04-21T10:26:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:07 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi026 to 5697M 2024-04-21T10:26:07.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:07 smithi169 ceph-mon[31469]: osdmap e49: 7 total, 7 up, 7 in 2024-04-21T10:26:07.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:07 smithi026 ceph-mon[27965]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:26:07.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:07 smithi026 ceph-mon[27965]: Detected new or changed devices on smithi026 2024-04-21T10:26:07.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:07 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi026 to 5697M 2024-04-21T10:26:07.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:07 smithi026 ceph-mon[27965]: osdmap e49: 7 total, 7 up, 7 in 2024-04-21T10:26:07.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:07 smithi190 ceph-mon[31423]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-21T10:26:07.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:07 smithi190 ceph-mon[31423]: Detected new or changed devices on smithi026 2024-04-21T10:26:07.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:07 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi026 to 5697M 2024-04-21T10:26:07.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:07 smithi190 ceph-mon[31423]: osdmap e49: 7 total, 7 up, 7 in 2024-04-21T10:26:07.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.817+0000 7f656f09a700 1 -- 172.21.15.169:0/2103775936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6568103550 msgr2=0x7f6568109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:07.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.817+0000 7f656f09a700 1 --2- 172.21.15.169:0/2103775936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6568103550 0x7f6568109e00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f656000b0d0 tx=0x7f656000b3e0 comp rx=0 tx=0).stop 2024-04-21T10:26:07.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.817+0000 7f656f09a700 1 -- 172.21.15.169:0/2103775936 shutdown_connections 2024-04-21T10:26:07.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.817+0000 7f656f09a700 1 --2- 172.21.15.169:0/2103775936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6568103550 0x7f6568109e00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:07.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.817+0000 7f656f09a700 1 --2- 172.21.15.169:0/2103775936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65681019b0 0x7f6568101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:07.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.817+0000 7f656f09a700 1 --2- 172.21.15.169:0/2103775936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6568102bb0 0x7f6568103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:07.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.817+0000 7f656f09a700 1 -- 172.21.15.169:0/2103775936 >> 172.21.15.169:0/2103775936 conn(0x7f65680fcf70 msgr2=0x7f65680ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:26:07.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.817+0000 7f656f09a700 1 -- 172.21.15.169:0/2103775936 shutdown_connections 2024-04-21T10:26:07.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.817+0000 7f656f09a700 1 -- 172.21.15.169:0/2103775936 wait complete. 2024-04-21T10:26:07.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f656f09a700 1 Processor -- start 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f656f09a700 1 -- start start 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f656f09a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65681019b0 0x7f6568198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f656f09a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6568102bb0 0x7f65681990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f656f09a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6568103550 0x7f65681a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f656f09a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f656810d480 con 0x7f6568102bb0 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f656f09a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f656810d300 con 0x7f65681019b0 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f656f09a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f656810d600 con 0x7f6568103550 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f656ce36700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65681019b0 0x7f6568198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f656ce36700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65681019b0 0x7f6568198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37046/0 (socket says 172.21.15.169:37046) 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f656ce36700 1 -- 172.21.15.169:0/548209213 learned_addr learned my addr 172.21.15.169:0/548209213 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f6567fff700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6568102bb0 0x7f65681990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.818+0000 7f656d637700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6568103550 0x7f65681a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.819+0000 7f656ce36700 1 -- 172.21.15.169:0/548209213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6568103550 msgr2=0x7f65681a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:07.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.819+0000 7f656ce36700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6568103550 0x7f65681a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:07.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.819+0000 7f656ce36700 1 -- 172.21.15.169:0/548209213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6568102bb0 msgr2=0x7f65681990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:07.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.819+0000 7f656ce36700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6568102bb0 0x7f65681990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:07.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.819+0000 7f656ce36700 1 -- 172.21.15.169:0/548209213 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65681a07e0 con 0x7f65681019b0 2024-04-21T10:26:07.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.819+0000 7f656d637700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6568103550 0x7f65681a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:26:07.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.819+0000 7f656ce36700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65681019b0 0x7f6568198b80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f655c00c990 tx=0x7f655c00cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:26:07.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.819+0000 7f6565ffb700 1 -- 172.21.15.169:0/548209213 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c007b00 con 0x7f65681019b0 2024-04-21T10:26:07.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.819+0000 7f656f09a700 1 -- 172.21.15.169:0/548209213 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65681a0ad0 con 0x7f65681019b0 2024-04-21T10:26:07.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.819+0000 7f6565ffb700 1 -- 172.21.15.169:0/548209213 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655c007ca0 con 0x7f65681019b0 2024-04-21T10:26:07.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.819+0000 7f6565ffb700 1 -- 172.21.15.169:0/548209213 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c018660 con 0x7f65681019b0 2024-04-21T10:26:07.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.819+0000 7f656f09a700 1 -- 172.21.15.169:0/548209213 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65681a1010 con 0x7f65681019b0 2024-04-21T10:26:07.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.820+0000 7f656f09a700 1 -- 172.21.15.169:0/548209213 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6568069730 con 0x7f65681019b0 2024-04-21T10:26:07.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.821+0000 7f6565ffb700 1 -- 172.21.15.169:0/548209213 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f655c00f450 con 0x7f65681019b0 2024-04-21T10:26:07.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.821+0000 7f6565ffb700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f65580a3c40 0x7f65580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:26:07.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.822+0000 7f6567fff700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f65580a3c40 0x7f65580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:26:07.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.822+0000 7f6565ffb700 1 -- 172.21.15.169:0/548209213 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7f655c0cf5d0 con 0x7f65681019b0 2024-04-21T10:26:07.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.822+0000 7f6567fff700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f65580a3c40 0x7f65580a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6554007be0 tx=0x7f65540073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:26:07.825 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:07.826+0000 7f6565ffb700 1 -- 172.21.15.169:0/548209213 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f655c014030 con 0x7f65681019b0 2024-04-21T10:26:08.031 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:08.031+0000 7f656f09a700 1 -- 172.21.15.169:0/548209213 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f65680ff0d0 con 0x7f65580a3c40 2024-04-21T10:26:08.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:08 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:26:08.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:08 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:26:08.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:08 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:08.640 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:08 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:26:08.640 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:08 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:26:08.640 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:08 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:08.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:08 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:26:08.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:08 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:26:08.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:08 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:09.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:09 smithi190 ceph-mon[31423]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:26:09.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:09 smithi190 ceph-mon[31423]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:09.634 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:09 smithi169 ceph-mon[31469]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:26:10.138 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:09 smithi169 ceph-mon[31469]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:10.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:09 smithi026 ceph-mon[27965]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:26:10.139 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:09 smithi026 ceph-mon[27965]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:11.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:11 smithi190 ceph-mon[31423]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:11.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:11 smithi190 ceph-mon[31423]: from='client.? 172.21.15.169:0/1382108345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "184da4b1-7943-425d-8e97-c89af799e656"}]: dispatch 2024-04-21T10:26:11.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:11 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "184da4b1-7943-425d-8e97-c89af799e656"}]: dispatch 2024-04-21T10:26:11.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:11 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "184da4b1-7943-425d-8e97-c89af799e656"}]': finished 2024-04-21T10:26:11.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:11 smithi190 ceph-mon[31423]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T10:26:11.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:11 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:11.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:11 smithi190 ceph-mon[31423]: from='client.? 172.21.15.169:0/3301966114' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:26:11.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:11 smithi026 ceph-mon[27965]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:11.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:11 smithi026 ceph-mon[27965]: from='client.? 172.21.15.169:0/1382108345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "184da4b1-7943-425d-8e97-c89af799e656"}]: dispatch 2024-04-21T10:26:11.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:11 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "184da4b1-7943-425d-8e97-c89af799e656"}]: dispatch 2024-04-21T10:26:11.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:11 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "184da4b1-7943-425d-8e97-c89af799e656"}]': finished 2024-04-21T10:26:11.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:11 smithi026 ceph-mon[27965]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T10:26:11.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:11 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:11.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:11 smithi026 ceph-mon[27965]: from='client.? 172.21.15.169:0/3301966114' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:26:11.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:11 smithi169 ceph-mon[31469]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:11.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:11 smithi169 ceph-mon[31469]: from='client.? 172.21.15.169:0/1382108345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "184da4b1-7943-425d-8e97-c89af799e656"}]: dispatch 2024-04-21T10:26:11.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:11 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "184da4b1-7943-425d-8e97-c89af799e656"}]: dispatch 2024-04-21T10:26:11.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:11 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "184da4b1-7943-425d-8e97-c89af799e656"}]': finished 2024-04-21T10:26:11.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:11 smithi169 ceph-mon[31469]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T10:26:11.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:11 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:11.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:11 smithi169 ceph-mon[31469]: from='client.? 172.21.15.169:0/3301966114' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:26:13.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:13 smithi190 ceph-mon[31423]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:13.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:13 smithi026 ceph-mon[27965]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:13.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:13 smithi169 ceph-mon[31469]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:15.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:15 smithi169 ceph-mon[31469]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:15.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:15 smithi190 ceph-mon[31423]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:15.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:15 smithi026 ceph-mon[27965]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:17.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:17 smithi190 ceph-mon[31423]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:17.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:17 smithi026 ceph-mon[27965]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:17.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:17 smithi169 ceph-mon[31469]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:18.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:18 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T10:26:18.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:18 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:18.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:18 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T10:26:18.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:18 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:18.546 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:18 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T10:26:18.547 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:18 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:19.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:19 smithi190 ceph-mon[31423]: Deploying daemon osd.7 on smithi169 2024-04-21T10:26:19.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:19 smithi190 ceph-mon[31423]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:19.537 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:19 smithi169 ceph-mon[31469]: Deploying daemon osd.7 on smithi169 2024-04-21T10:26:19.537 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:19 smithi169 ceph-mon[31469]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:19.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:19 smithi026 ceph-mon[27965]: Deploying daemon osd.7 on smithi169 2024-04-21T10:26:19.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:19 smithi026 ceph-mon[27965]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:21.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:21 smithi169 ceph-mon[31469]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:21.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:21 smithi190 ceph-mon[31423]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:21.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:21 smithi026 ceph-mon[27965]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:23.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:23 smithi190 ceph-mon[31423]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:23 smithi026 ceph-mon[27965]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:23.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:23 smithi169 ceph-mon[31469]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:24.545 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:24 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:24.545 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:24 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:24.545 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:24 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:24.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:24 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:24.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:24 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:24.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:24 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:24.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:24 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:24.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:24 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:24.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:24 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:25.541 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:25 smithi169 ceph-mon[31469]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:25.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:25 smithi026 ceph-mon[27965]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:25.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:25 smithi190 ceph-mon[31423]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:26.996 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.997+0000 7f6565ffb700 1 -- 172.21.15.169:0/548209213 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f65680ff0d0 con 0x7f65580a3c40 2024-04-21T10:26:26.997 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 7 on host 'smithi169' 2024-04-21T10:26:26.999 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.999+0000 7f656f09a700 1 -- 172.21.15.169:0/548209213 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f65580a3c40 msgr2=0x7f65580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:26.999 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.999+0000 7f656f09a700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f65580a3c40 0x7f65580a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6554007be0 tx=0x7f65540073d0 comp rx=0 tx=0).stop 2024-04-21T10:26:26.999 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.999+0000 7f656f09a700 1 -- 172.21.15.169:0/548209213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65681019b0 msgr2=0x7f6568198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:26.999 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.999+0000 7f656f09a700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65681019b0 0x7f6568198b80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f655c00c990 tx=0x7f655c00cca0 comp rx=0 tx=0).stop 2024-04-21T10:26:26.999 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.999+0000 7f656f09a700 1 -- 172.21.15.169:0/548209213 shutdown_connections 2024-04-21T10:26:27.000 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.999+0000 7f656f09a700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f65580a3c40 0x7f65580a6100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:27.000 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.999+0000 7f656f09a700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65681019b0 0x7f6568198b80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:27.000 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.999+0000 7f656f09a700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6568103550 0x7f65681a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:27.000 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.999+0000 7f656f09a700 1 --2- 172.21.15.169:0/548209213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6568102bb0 0x7f65681990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:27.000 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.999+0000 7f656f09a700 1 -- 172.21.15.169:0/548209213 >> 172.21.15.169:0/548209213 conn(0x7f65680fcf70 msgr2=0x7f65680fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:26:27.000 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.999+0000 7f656f09a700 1 -- 172.21.15.169:0/548209213 shutdown_connections 2024-04-21T10:26:27.000 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:26:26.999+0000 7f656f09a700 1 -- 172.21.15.169:0/548209213 wait complete. 2024-04-21T10:26:27.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:27 smithi190 ceph-mon[31423]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:27.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:27.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:27.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:27.591 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:27 smithi169 ceph-mon[31469]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:27.591 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.591 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.591 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:27.591 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:27.591 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.591 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.591 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.592 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:27.623 DEBUG:teuthology.orchestra.run.smithi169:osd.7> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.7.service 2024-04-21T10:26:27.626 INFO:tasks.cephadm:Deploying osd.8 on smithi190 with /dev/vg_nvme/lv_2... 2024-04-21T10:26:27.626 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T10:26:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:27 smithi026 ceph-mon[27965]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:27.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:27.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:27.873 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:26:28.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:28 smithi190 ceph-mon[31423]: from='osd.7 [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T10:26:28.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:28 smithi190 ceph-mon[31423]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T10:26:28.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:28 smithi026 ceph-mon[27965]: from='osd.7 [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T10:26:28.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:28 smithi026 ceph-mon[27965]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T10:26:28.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:28 smithi169 ceph-mon[31469]: from='osd.7 [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T10:26:28.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:28 smithi169 ceph-mon[31469]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T10:26:29.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:29 smithi169 ceph-mon[31469]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:29.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:29 smithi169 ceph-mon[31469]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T10:26:29.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:29 smithi169 ceph-mon[31469]: from='osd.7 [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:26:29.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:29 smithi169 ceph-mon[31469]: osdmap e51: 8 total, 7 up, 8 in 2024-04-21T10:26:29.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:29.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:29 smithi169 ceph-mon[31469]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:26:29.449 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:26:29 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:26:29.268+0000 7f24f3e6e700 -1 osd.7 0 waiting for initial osdmap 2024-04-21T10:26:29.449 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:26:29 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:26:29.278+0000 7f24efc61700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T10:26:29.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:29 smithi190 ceph-mon[31423]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:29.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:29 smithi190 ceph-mon[31423]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T10:26:29.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:29 smithi190 ceph-mon[31423]: from='osd.7 [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:26:29.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:29 smithi190 ceph-mon[31423]: osdmap e51: 8 total, 7 up, 8 in 2024-04-21T10:26:29.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:29.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:29 smithi190 ceph-mon[31423]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:26:29.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:29 smithi026 ceph-mon[27965]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:29.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:29 smithi026 ceph-mon[27965]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T10:26:29.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:29 smithi026 ceph-mon[27965]: from='osd.7 [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:26:29.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:29 smithi026 ceph-mon[27965]: osdmap e51: 8 total, 7 up, 8 in 2024-04-21T10:26:29.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:29.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:29 smithi026 ceph-mon[27965]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:26:29.943 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T10:26:29.943 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T10:26:29.943 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-04-21T10:26:29.943 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-04-21T10:26:29.943 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113723 s, 922 MB/s 2024-04-21T10:26:29.943 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-04-21T10:26:30.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:30 smithi169 ceph-mon[31469]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-21T10:26:30.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:30 smithi169 ceph-mon[31469]: osdmap e52: 8 total, 7 up, 8 in 2024-04-21T10:26:30.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:30.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:30 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:30.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:30 smithi190 ceph-mon[31423]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-21T10:26:30.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:30 smithi190 ceph-mon[31423]: osdmap e52: 8 total, 7 up, 8 in 2024-04-21T10:26:30.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:30.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:30 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:30.563 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_2 2024-04-21T10:26:30.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:30 smithi026 ceph-mon[27965]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-21T10:26:30.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:30 smithi026 ceph-mon[27965]: osdmap e52: 8 total, 7 up, 8 in 2024-04-21T10:26:30.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:30.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:30 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:30.814 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:26:31.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: purged_snaps scrub starts 2024-04-21T10:26:31.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: purged_snaps scrub ok 2024-04-21T10:26:31.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:31.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: osd.7 [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185] boot 2024-04-21T10:26:31.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T10:26:31.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:31.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: Detected new or changed devices on smithi169 2024-04-21T10:26:31.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:31.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:31.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:31.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:31.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:31.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-21T10:26:31.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:31.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:31.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:31.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:31.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:31 smithi190 ceph-mon[31423]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T10:26:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: purged_snaps scrub starts 2024-04-21T10:26:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: purged_snaps scrub ok 2024-04-21T10:26:31.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:31.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: osd.7 [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185] boot 2024-04-21T10:26:31.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T10:26:31.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:31.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: Detected new or changed devices on smithi169 2024-04-21T10:26:31.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:31.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:31.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:31.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:31.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:31.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-21T10:26:31.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:31.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:31.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:31.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:31.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:31 smithi026 ceph-mon[27965]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T10:26:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: purged_snaps scrub starts 2024-04-21T10:26:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: purged_snaps scrub ok 2024-04-21T10:26:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-21T10:26:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: osd.7 [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185] boot 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: Detected new or changed devices on smithi169 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:31.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:31 smithi169 ceph-mon[31469]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T10:26:32.407 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.405+0000 7f5394ae9700 1 -- 172.21.15.190:0/765983490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53901087a0 msgr2=0x7f539010ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.405+0000 7f5394ae9700 1 --2- 172.21.15.190:0/765983490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53901087a0 0x7f539010ac10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f538800b0a0 tx=0x7f538800b3b0 comp rx=0 tx=0).stop 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.406+0000 7f5394ae9700 1 -- 172.21.15.190:0/765983490 shutdown_connections 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.406+0000 7f5394ae9700 1 --2- 172.21.15.190:0/765983490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53901087a0 0x7f539010ac10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.406+0000 7f5394ae9700 1 --2- 172.21.15.190:0/765983490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5390103540 0x7f5390105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.406+0000 7f5394ae9700 1 --2- 172.21.15.190:0/765983490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5390105e70 0x7f5390108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.406+0000 7f5394ae9700 1 -- 172.21.15.190:0/765983490 >> 172.21.15.190:0/765983490 conn(0x7f53900fcf70 msgr2=0x7f53900ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.406+0000 7f5394ae9700 1 -- 172.21.15.190:0/765983490 shutdown_connections 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.406+0000 7f5394ae9700 1 -- 172.21.15.190:0/765983490 wait complete. 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.406+0000 7f5394ae9700 1 Processor -- start 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.406+0000 7f5394ae9700 1 -- start start 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f5394ae9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5390103540 0x7f5390198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f5394ae9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5390105e70 0x7f5390199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f5394ae9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53901087a0 0x7f53901a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:26:32.408 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f5394ae9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f539010e540 con 0x7f5390105e70 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f5394ae9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f539010e3c0 con 0x7f53901087a0 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f5394ae9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f539010e6c0 con 0x7f5390103540 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f538e59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5390103540 0x7f5390198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f538e59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5390103540 0x7f5390198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:50682/0 (socket says 172.21.15.190:50682) 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f538e59c700 1 -- 172.21.15.190:0/3201275449 learned_addr learned my addr 172.21.15.190:0/3201275449 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f5387fff700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5390105e70 0x7f5390199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f538ed9d700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53901087a0 0x7f53901a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f538e59c700 1 -- 172.21.15.190:0/3201275449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53901087a0 msgr2=0x7f53901a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f538e59c700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53901087a0 0x7f53901a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f538e59c700 1 -- 172.21.15.190:0/3201275449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5390105e70 msgr2=0x7f5390199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f538e59c700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5390105e70 0x7f5390199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f538e59c700 1 -- 172.21.15.190:0/3201275449 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53901a0860 con 0x7f5390103540 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f538ed9d700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53901087a0 0x7f53901a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:26:32.409 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f5387fff700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5390105e70 0x7f5390199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:26:32.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f538e59c700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5390103540 0x7f5390198c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f538000d950 tx=0x7f538000dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:26:32.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f53877fe700 1 -- 172.21.15.190:0/3201275449 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380016070 con 0x7f5390103540 2024-04-21T10:26:32.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f53877fe700 1 -- 172.21.15.190:0/3201275449 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5380010460 con 0x7f5390103540 2024-04-21T10:26:32.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f5394ae9700 1 -- 172.21.15.190:0/3201275449 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53901a0b50 con 0x7f5390103540 2024-04-21T10:26:32.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.407+0000 7f53877fe700 1 -- 172.21.15.190:0/3201275449 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538000f620 con 0x7f5390103540 2024-04-21T10:26:32.410 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.408+0000 7f5394ae9700 1 -- 172.21.15.190:0/3201275449 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53901a1090 con 0x7f5390103540 2024-04-21T10:26:32.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.409+0000 7f5394ae9700 1 -- 172.21.15.190:0/3201275449 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5390069730 con 0x7f5390103540 2024-04-21T10:26:32.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.410+0000 7f53877fe700 1 -- 172.21.15.190:0/3201275449 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5380019070 con 0x7f5390103540 2024-04-21T10:26:32.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.410+0000 7f53877fe700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f537c0a3c40 0x7f537c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:26:32.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.410+0000 7f53877fe700 1 -- 172.21.15.190:0/3201275449 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5426+0+0 (secure 0 0 0) 0x7f53800cf530 con 0x7f5390103540 2024-04-21T10:26:32.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.410+0000 7f5387fff700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f537c0a3c40 0x7f537c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:26:32.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.411+0000 7f5387fff700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f537c0a3c40 0x7f537c0a6100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f537800bcf0 tx=0x7f537800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:26:32.414 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.415+0000 7f53877fe700 1 -- 172.21.15.190:0/3201275449 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f538009ca50 con 0x7f5390103540 2024-04-21T10:26:32.620 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:32.620+0000 7f5394ae9700 1 -- 172.21.15.190:0/3201275449 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5390062e80 con 0x7f537c0a3c40 2024-04-21T10:26:33.513 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:33 smithi190 ceph-mon[31423]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:33.513 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:33 smithi190 ceph-mon[31423]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T10:26:33.513 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:33 smithi190 ceph-mon[31423]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:26:33.513 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:33 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:26:33.514 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:33 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:26:33.514 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:33 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:33 smithi026 ceph-mon[27965]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:33 smithi026 ceph-mon[27965]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T10:26:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:33 smithi026 ceph-mon[27965]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:26:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:26:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:26:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:33 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:33 smithi169 ceph-mon[31469]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:33 smithi169 ceph-mon[31469]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T10:26:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:33 smithi169 ceph-mon[31469]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:26:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:33 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:26:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:33 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:26:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:33 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:34.401 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:34 smithi190 ceph-mon[31423]: osdmap e56: 8 total, 8 up, 8 in 2024-04-21T10:26:34.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:34 smithi026 ceph-mon[27965]: osdmap e56: 8 total, 8 up, 8 in 2024-04-21T10:26:34.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:34 smithi169 ceph-mon[31469]: osdmap e56: 8 total, 8 up, 8 in 2024-04-21T10:26:35.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:35 smithi190 ceph-mon[31423]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:35.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:35 smithi190 ceph-mon[31423]: osdmap e57: 8 total, 8 up, 8 in 2024-04-21T10:26:35.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:35 smithi190 ceph-mon[31423]: from='client.? 172.21.15.190:0/2090981372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da5499d-cd87-4182-87f7-6caffa3f5329"}]: dispatch 2024-04-21T10:26:35.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:35 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da5499d-cd87-4182-87f7-6caffa3f5329"}]: dispatch 2024-04-21T10:26:35.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:35 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da5499d-cd87-4182-87f7-6caffa3f5329"}]': finished 2024-04-21T10:26:35.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:35 smithi190 ceph-mon[31423]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T10:26:35.551 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:35 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:35.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:35 smithi026 ceph-mon[27965]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:35.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:35 smithi026 ceph-mon[27965]: osdmap e57: 8 total, 8 up, 8 in 2024-04-21T10:26:35.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:35 smithi026 ceph-mon[27965]: from='client.? 172.21.15.190:0/2090981372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da5499d-cd87-4182-87f7-6caffa3f5329"}]: dispatch 2024-04-21T10:26:35.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:35 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da5499d-cd87-4182-87f7-6caffa3f5329"}]: dispatch 2024-04-21T10:26:35.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:35 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da5499d-cd87-4182-87f7-6caffa3f5329"}]': finished 2024-04-21T10:26:35.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:35 smithi026 ceph-mon[27965]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T10:26:35.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:35 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:35 smithi169 ceph-mon[31469]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:35 smithi169 ceph-mon[31469]: osdmap e57: 8 total, 8 up, 8 in 2024-04-21T10:26:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:35 smithi169 ceph-mon[31469]: from='client.? 172.21.15.190:0/2090981372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da5499d-cd87-4182-87f7-6caffa3f5329"}]: dispatch 2024-04-21T10:26:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:35 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da5499d-cd87-4182-87f7-6caffa3f5329"}]: dispatch 2024-04-21T10:26:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:35 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da5499d-cd87-4182-87f7-6caffa3f5329"}]': finished 2024-04-21T10:26:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:35 smithi169 ceph-mon[31469]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T10:26:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:35 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:36.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:36 smithi026 ceph-mon[27965]: from='client.? 172.21.15.190:0/3758826327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:26:36.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:36 smithi169 ceph-mon[31469]: from='client.? 172.21.15.190:0/3758826327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:26:36.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:36 smithi190 ceph-mon[31423]: from='client.? 172.21.15.190:0/3758826327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:26:37.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:37 smithi026 ceph-mon[27965]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:37.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:37 smithi169 ceph-mon[31469]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:37.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:37 smithi190 ceph-mon[31423]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:39.432 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:39 smithi190 ceph-mon[31423]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:39.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:39 smithi026 ceph-mon[27965]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:39.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:39 smithi169 ceph-mon[31469]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:41.543 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:41 smithi190 ceph-mon[31423]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:41 smithi026 ceph-mon[27965]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:41.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:41 smithi169 ceph-mon[31469]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:43.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:43 smithi190 ceph-mon[31423]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:43.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T10:26:43.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:43.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:43 smithi190 ceph-mon[31423]: Deploying daemon osd.8 on smithi190 2024-04-21T10:26:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:43 smithi026 ceph-mon[27965]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T10:26:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:43 smithi026 ceph-mon[27965]: Deploying daemon osd.8 on smithi190 2024-04-21T10:26:43.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:43 smithi169 ceph-mon[31469]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:43.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T10:26:43.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:43.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:43 smithi169 ceph-mon[31469]: Deploying daemon osd.8 on smithi190 2024-04-21T10:26:45.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:45 smithi026 ceph-mon[27965]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:45.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:45 smithi169 ceph-mon[31469]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:45.773 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:45 smithi190 ceph-mon[31423]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:47.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:47 smithi190 ceph-mon[31423]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:47.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:47 smithi026 ceph-mon[27965]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:47.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:47 smithi169 ceph-mon[31469]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:49 smithi026 ceph-mon[27965]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:49.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:49 smithi169 ceph-mon[31469]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:49.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:49.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:49.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:49.772 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:49 smithi190 ceph-mon[31423]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:49.773 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:49.773 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:49.773 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:51.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:51 smithi026 ceph-mon[27965]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:51.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:51 smithi169 ceph-mon[31469]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:51.757 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:51 smithi190 ceph-mon[31423]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:51.842 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 8 on host 'smithi190' 2024-04-21T10:26:51.842 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.841+0000 7f53877fe700 1 -- 172.21.15.190:0/3201275449 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5390062e80 con 0x7f537c0a3c40 2024-04-21T10:26:51.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.843+0000 7f5394ae9700 1 -- 172.21.15.190:0/3201275449 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f537c0a3c40 msgr2=0x7f537c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:51.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.843+0000 7f5394ae9700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f537c0a3c40 0x7f537c0a6100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f537800bcf0 tx=0x7f537800b2c0 comp rx=0 tx=0).stop 2024-04-21T10:26:51.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.843+0000 7f5394ae9700 1 -- 172.21.15.190:0/3201275449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5390103540 msgr2=0x7f5390198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:51.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.843+0000 7f5394ae9700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5390103540 0x7f5390198c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f538000d950 tx=0x7f538000dd10 comp rx=0 tx=0).stop 2024-04-21T10:26:51.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.843+0000 7f5394ae9700 1 -- 172.21.15.190:0/3201275449 shutdown_connections 2024-04-21T10:26:51.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.843+0000 7f5394ae9700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f537c0a3c40 0x7f537c0a6100 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:51.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.843+0000 7f5394ae9700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53901087a0 0x7f53901a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:51.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.843+0000 7f5394ae9700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5390103540 0x7f5390198c10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:51.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.843+0000 7f5394ae9700 1 --2- 172.21.15.190:0/3201275449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5390105e70 0x7f5390199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:51.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.843+0000 7f5394ae9700 1 -- 172.21.15.190:0/3201275449 >> 172.21.15.190:0/3201275449 conn(0x7f53900fcf70 msgr2=0x7f5390103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:26:51.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.843+0000 7f5394ae9700 1 -- 172.21.15.190:0/3201275449 shutdown_connections 2024-04-21T10:26:51.845 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:26:51.843+0000 7f5394ae9700 1 -- 172.21.15.190:0/3201275449 wait complete. 2024-04-21T10:26:52.453 DEBUG:teuthology.orchestra.run.smithi190:osd.8> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.8.service 2024-04-21T10:26:52.455 INFO:tasks.cephadm:Deploying osd.9 on smithi026 with /dev/vg_nvme/lv_1... 2024-04-21T10:26:52.455 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T10:26:52.720 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:26:53.063 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:52 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:53.063 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:52 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:53.063 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:52 smithi026 ceph-mon[27965]: from='osd.8 [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T10:26:53.063 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:52 smithi026 ceph-mon[27965]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T10:26:53.063 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:52 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:53.063 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:52 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:53.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:52 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:53.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:52 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:53.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:52 smithi169 ceph-mon[31469]: from='osd.8 [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T10:26:53.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:52 smithi169 ceph-mon[31469]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T10:26:53.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:52 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:53.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:52 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:53.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:52 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:53.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:52 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:53.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:52 smithi190 ceph-mon[31423]: from='osd.8 [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T10:26:53.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:52 smithi190 ceph-mon[31423]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T10:26:53.229 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:52 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:53.230 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:52 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:54.168 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:53 smithi026 ceph-mon[27965]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:54.168 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:53 smithi026 ceph-mon[27965]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T10:26:54.168 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:53 smithi026 ceph-mon[27965]: osdmap e59: 9 total, 8 up, 9 in 2024-04-21T10:26:54.168 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:53 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:54.168 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:53 smithi026 ceph-mon[27965]: from='osd.8 [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:26:54.168 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:53 smithi026 ceph-mon[27965]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:26:54.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:53 smithi169 ceph-mon[31469]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:54.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:53 smithi169 ceph-mon[31469]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T10:26:54.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:53 smithi169 ceph-mon[31469]: osdmap e59: 9 total, 8 up, 9 in 2024-04-21T10:26:54.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:53 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:54.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:53 smithi169 ceph-mon[31469]: from='osd.8 [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:26:54.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:53 smithi169 ceph-mon[31469]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:26:54.216 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:53 smithi190 ceph-mon[31423]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:54.216 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:53 smithi190 ceph-mon[31423]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T10:26:54.216 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:53 smithi190 ceph-mon[31423]: osdmap e59: 9 total, 8 up, 9 in 2024-04-21T10:26:54.216 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:53 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:54.216 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:53 smithi190 ceph-mon[31423]: from='osd.8 [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:26:54.216 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:53 smithi190 ceph-mon[31423]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:26:54.217 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:26:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:26:53.806+0000 7fddc068f700 -1 osd.8 0 waiting for initial osdmap 2024-04-21T10:26:54.217 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:26:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:26:53.817+0000 7fddbc482700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T10:26:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T10:26:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T10:26:54.750 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-04-21T10:26:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-04-21T10:26:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00825115 s, 1.3 GB/s 2024-04-21T10:26:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-04-21T10:26:55.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:54 smithi026 ceph-mon[27965]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-04-21T10:26:55.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:54 smithi026 ceph-mon[27965]: osdmap e60: 9 total, 8 up, 9 in 2024-04-21T10:26:55.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:54 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:55.076 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:54 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:55.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:54 smithi169 ceph-mon[31469]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-04-21T10:26:55.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:54 smithi169 ceph-mon[31469]: osdmap e60: 9 total, 8 up, 9 in 2024-04-21T10:26:55.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:55.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:55.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:54 smithi190 ceph-mon[31423]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-04-21T10:26:55.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:54 smithi190 ceph-mon[31423]: osdmap e60: 9 total, 8 up, 9 in 2024-04-21T10:26:55.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:54 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:55.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:54 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:55.474 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_1 2024-04-21T10:26:55.725 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:26:56.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:55 smithi190 ceph-mon[31423]: purged_snaps scrub starts 2024-04-21T10:26:56.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:55 smithi190 ceph-mon[31423]: purged_snaps scrub ok 2024-04-21T10:26:56.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:55 smithi190 ceph-mon[31423]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:56.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:55 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:56.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:55 smithi190 ceph-mon[31423]: osd.8 [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578] boot 2024-04-21T10:26:56.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:55 smithi190 ceph-mon[31423]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T10:26:56.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:55 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:56.098 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:55 smithi026 ceph-mon[27965]: purged_snaps scrub starts 2024-04-21T10:26:56.098 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:55 smithi026 ceph-mon[27965]: purged_snaps scrub ok 2024-04-21T10:26:56.098 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:55 smithi026 ceph-mon[27965]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:56.098 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:55 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:56.098 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:55 smithi026 ceph-mon[27965]: osd.8 [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578] boot 2024-04-21T10:26:56.098 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:55 smithi026 ceph-mon[27965]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T10:26:56.098 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:55 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:56.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:55 smithi169 ceph-mon[31469]: purged_snaps scrub starts 2024-04-21T10:26:56.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:55 smithi169 ceph-mon[31469]: purged_snaps scrub ok 2024-04-21T10:26:56.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:55 smithi169 ceph-mon[31469]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-21T10:26:56.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:55 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:56.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:55 smithi169 ceph-mon[31469]: osd.8 [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578] boot 2024-04-21T10:26:56.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:55 smithi169 ceph-mon[31469]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T10:26:56.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:55 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:26:56.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T10:26:56.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:56.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:56.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:56.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:56.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:56.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:56.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:56.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:56.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:56.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:56.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:56.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:56.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:56 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:57.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T10:26:57.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:57.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:57.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:57.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:57.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:57.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:57.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:57.200 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:57.200 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:57.200 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:57.200 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:57.200 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:57.201 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:56 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T10:26:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:26:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:57.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:26:57.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:57.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:26:57.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:56 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:26:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.525+0000 7fd2113c5700 1 -- 172.21.15.26:0/3771277555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd20c103cc0 msgr2=0x7fd20c104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.525+0000 7fd2113c5700 1 --2- 172.21.15.26:0/3771277555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd20c103cc0 0x7fd20c104120 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd2000099c0 tx=0x7fd200009cd0 comp rx=0 tx=0).stop 2024-04-21T10:26:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.525+0000 7fd2113c5700 1 -- 172.21.15.26:0/3771277555 shutdown_connections 2024-04-21T10:26:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.525+0000 7fd2113c5700 1 --2- 172.21.15.26:0/3771277555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20c102ac0 0x7fd20c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.525+0000 7fd2113c5700 1 --2- 172.21.15.26:0/3771277555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd20c104660 0x7fd20c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.525+0000 7fd2113c5700 1 --2- 172.21.15.26:0/3771277555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd20c103cc0 0x7fd20c104120 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.525+0000 7fd2113c5700 1 -- 172.21.15.26:0/3771277555 >> 172.21.15.26:0/3771277555 conn(0x7fd20c0fe060 msgr2=0x7fd20c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.525+0000 7fd2113c5700 1 -- 172.21.15.26:0/3771277555 shutdown_connections 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.525+0000 7fd2113c5700 1 -- 172.21.15.26:0/3771277555 wait complete. 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd2113c5700 1 Processor -- start 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd2113c5700 1 -- start start 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd2113c5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20c102ac0 0x7fd20c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd2113c5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd20c103cc0 0x7fd20c19b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd2113c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd20c104660 0x7fd20c1a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd2113c5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c10e530 con 0x7fd20c104660 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd2113c5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c10e3b0 con 0x7fd20c102ac0 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd2113c5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c10e6b0 con 0x7fd20c103cc0 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd20b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd20c104660 0x7fd20c1a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd20affd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20c102ac0 0x7fd20c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd20b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd20c104660 0x7fd20c1a2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48698/0 (socket says 172.21.15.26:48698) 2024-04-21T10:26:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd20b7fe700 1 -- 172.21.15.26:0/3647699222 learned_addr learned my addr 172.21.15.26:0/3647699222 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:26:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.526+0000 7fd20a7fc700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd20c103cc0 0x7fd20c19b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:26:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.527+0000 7fd20b7fe700 1 -- 172.21.15.26:0/3647699222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd20c103cc0 msgr2=0x7fd20c19b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.527+0000 7fd20b7fe700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd20c103cc0 0x7fd20c19b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.527+0000 7fd20b7fe700 1 -- 172.21.15.26:0/3647699222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20c102ac0 msgr2=0x7fd20c19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:26:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.527+0000 7fd20b7fe700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20c102ac0 0x7fd20c19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:26:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.527+0000 7fd20b7fe700 1 -- 172.21.15.26:0/3647699222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20c1a2920 con 0x7fd20c104660 2024-04-21T10:26:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.527+0000 7fd20b7fe700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd20c104660 0x7fd20c1a2250 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd204007d70 tx=0x7fd20400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:26:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.527+0000 7fd1f3fff700 1 -- 172.21.15.26:0/3647699222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204016020 con 0x7fd20c104660 2024-04-21T10:26:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.527+0000 7fd1f3fff700 1 -- 172.21.15.26:0/3647699222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd20400ac90 con 0x7fd20c104660 2024-04-21T10:26:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.527+0000 7fd2113c5700 1 -- 172.21.15.26:0/3647699222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20c1a2c10 con 0x7fd20c104660 2024-04-21T10:26:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.527+0000 7fd2113c5700 1 -- 172.21.15.26:0/3647699222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20c1a30d0 con 0x7fd20c104660 2024-04-21T10:26:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.527+0000 7fd1f3fff700 1 -- 172.21.15.26:0/3647699222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2040115e0 con 0x7fd20c104660 2024-04-21T10:26:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.528+0000 7fd2113c5700 1 -- 172.21.15.26:0/3647699222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd20c069730 con 0x7fd20c104660 2024-04-21T10:26:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.529+0000 7fd1f3fff700 1 -- 172.21.15.26:0/3647699222 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd204019020 con 0x7fd20c104660 2024-04-21T10:26:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.530+0000 7fd1f3fff700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd1f40a3c40 0x7fd1f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:26:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.530+0000 7fd20affd700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd1f40a3c40 0x7fd1f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:26:57.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.530+0000 7fd1f3fff700 1 -- 172.21.15.26:0/3647699222 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5858+0+0 (secure 0 0 0) 0x7fd2040cf9f0 con 0x7fd20c104660 2024-04-21T10:26:57.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.531+0000 7fd20affd700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd1f40a3c40 0x7fd1f40a6100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd20c19bb80 tx=0x7fd1fc00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:26:57.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.535+0000 7fd1f3fff700 1 -- 172.21.15.26:0/3647699222 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd204020020 con 0x7fd20c104660 2024-04-21T10:26:57.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:26:57.756+0000 7fd2113c5700 1 -- 172.21.15.26:0/3647699222 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd20c076390 con 0x7fd1f40a3c40 2024-04-21T10:26:58.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:57 smithi026 ceph-mon[27965]: Detected new or changed devices on smithi190 2024-04-21T10:26:58.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:57 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi190 to 5696M 2024-04-21T10:26:58.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:57 smithi026 ceph-mon[27965]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:26:58.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:57 smithi026 ceph-mon[27965]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T10:26:58.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:57 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:26:58.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:57 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:26:58.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:57 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:58.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:57 smithi169 ceph-mon[31469]: Detected new or changed devices on smithi190 2024-04-21T10:26:58.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:57 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi190 to 5696M 2024-04-21T10:26:58.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:57 smithi169 ceph-mon[31469]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:26:58.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:57 smithi169 ceph-mon[31469]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T10:26:58.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:57 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:26:58.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:57 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:26:58.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:57 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:58.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:57 smithi190 ceph-mon[31423]: Detected new or changed devices on smithi190 2024-04-21T10:26:58.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:57 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi190 to 5696M 2024-04-21T10:26:58.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:57 smithi190 ceph-mon[31423]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:26:58.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:57 smithi190 ceph-mon[31423]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T10:26:58.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:57 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:26:58.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:57 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:26:58.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:57 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:26:58.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:58 smithi026 ceph-mon[27965]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:26:58.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:58 smithi026 ceph-mon[27965]: osdmap e64: 9 total, 9 up, 9 in 2024-04-21T10:26:59.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:58 smithi169 ceph-mon[31469]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:26:59.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:58 smithi169 ceph-mon[31469]: osdmap e64: 9 total, 9 up, 9 in 2024-04-21T10:26:59.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:58 smithi190 ceph-mon[31423]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:26:59.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:58 smithi190 ceph-mon[31423]: osdmap e64: 9 total, 9 up, 9 in 2024-04-21T10:27:00.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:59 smithi026 ceph-mon[27965]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:00.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:26:59 smithi026 ceph-mon[27965]: osdmap e65: 9 total, 9 up, 9 in 2024-04-21T10:27:00.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:59 smithi169 ceph-mon[31469]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:00.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:26:59 smithi169 ceph-mon[31469]: osdmap e65: 9 total, 9 up, 9 in 2024-04-21T10:27:00.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:59 smithi190 ceph-mon[31423]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:00.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:26:59 smithi190 ceph-mon[31423]: osdmap e65: 9 total, 9 up, 9 in 2024-04-21T10:27:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:00 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1202067423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47e78d57-1332-4623-a4cf-dcd4d4af2945"}]: dispatch 2024-04-21T10:27:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:00 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1202067423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47e78d57-1332-4623-a4cf-dcd4d4af2945"}]': finished 2024-04-21T10:27:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:00 smithi026 ceph-mon[27965]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T10:27:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:01.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:00 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1202067423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47e78d57-1332-4623-a4cf-dcd4d4af2945"}]: dispatch 2024-04-21T10:27:01.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:00 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1202067423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47e78d57-1332-4623-a4cf-dcd4d4af2945"}]': finished 2024-04-21T10:27:01.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:00 smithi169 ceph-mon[31469]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T10:27:01.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:00 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:00 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1202067423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47e78d57-1332-4623-a4cf-dcd4d4af2945"}]: dispatch 2024-04-21T10:27:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:00 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1202067423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47e78d57-1332-4623-a4cf-dcd4d4af2945"}]': finished 2024-04-21T10:27:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:00 smithi190 ceph-mon[31423]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T10:27:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:01 smithi026 ceph-mon[27965]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:01 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1021896773' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:27:02.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:01 smithi169 ceph-mon[31469]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:02.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:01 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1021896773' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:27:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:01 smithi190 ceph-mon[31423]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:02.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:01 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1021896773' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:27:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:03 smithi026 ceph-mon[27965]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-21T10:27:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:03 smithi169 ceph-mon[31469]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-21T10:27:04.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:03 smithi190 ceph-mon[31423]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-21T10:27:06.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:05 smithi026 ceph-mon[27965]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-21T10:27:06.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:05 smithi169 ceph-mon[31469]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-21T10:27:06.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:05 smithi190 ceph-mon[31423]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-21T10:27:08.172 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:07 smithi026 ceph-mon[27965]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-21T10:27:08.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:07 smithi169 ceph-mon[31469]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-21T10:27:08.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:07 smithi190 ceph-mon[31423]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-21T10:27:09.158 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:08 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T10:27:09.158 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:08 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:09.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:08 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T10:27:09.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:08 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:09.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:08 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T10:27:09.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:08 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:10.162 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:09 smithi026 ceph-mon[27965]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-21T10:27:10.163 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:09 smithi026 ceph-mon[27965]: Deploying daemon osd.9 on smithi026 2024-04-21T10:27:10.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:09 smithi169 ceph-mon[31469]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-21T10:27:10.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:09 smithi169 ceph-mon[31469]: Deploying daemon osd.9 on smithi026 2024-04-21T10:27:10.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:09 smithi190 ceph-mon[31423]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-21T10:27:10.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:09 smithi190 ceph-mon[31423]: Deploying daemon osd.9 on smithi026 2024-04-21T10:27:12.135 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:11 smithi026 ceph-mon[27965]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T10:27:12.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:11 smithi169 ceph-mon[31469]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T10:27:12.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:11 smithi190 ceph-mon[31423]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T10:27:13.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:13 smithi026 ceph-mon[27965]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:14.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:13 smithi169 ceph-mon[31469]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:14.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:13 smithi190 ceph-mon[31423]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:15.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:15 smithi190 ceph-mon[31423]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:15.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:15.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:15.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:27:15.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:15 smithi026 ceph-mon[27965]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:15.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:15.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:15.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:27:15.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:15 smithi169 ceph-mon[31469]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:15.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:15.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:15.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:27:17.390 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:17 smithi026 ceph-mon[27965]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:17.391 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:17 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:17.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:17 smithi190 ceph-mon[31423]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:17.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:17 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:17.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:17 smithi169 ceph-mon[31469]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:17.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:17 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.169+0000 7fd1f3fff700 1 -- 172.21.15.26:0/3647699222 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd20c076390 con 0x7fd1f40a3c40 2024-04-21T10:27:18.170 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 9 on host 'smithi026' 2024-04-21T10:27:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.171+0000 7fd2113c5700 1 -- 172.21.15.26:0/3647699222 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd1f40a3c40 msgr2=0x7fd1f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:27:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.171+0000 7fd2113c5700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd1f40a3c40 0x7fd1f40a6100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd20c19bb80 tx=0x7fd1fc00a3b0 comp rx=0 tx=0).stop 2024-04-21T10:27:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.172+0000 7fd2113c5700 1 -- 172.21.15.26:0/3647699222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd20c104660 msgr2=0x7fd20c1a2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:27:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.172+0000 7fd2113c5700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd20c104660 0x7fd20c1a2250 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd204007d70 tx=0x7fd20400a340 comp rx=0 tx=0).stop 2024-04-21T10:27:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.172+0000 7fd2113c5700 1 -- 172.21.15.26:0/3647699222 shutdown_connections 2024-04-21T10:27:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.172+0000 7fd2113c5700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd1f40a3c40 0x7fd1f40a6100 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.172+0000 7fd2113c5700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20c102ac0 0x7fd20c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.172+0000 7fd2113c5700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd20c104660 0x7fd20c1a2250 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.172+0000 7fd2113c5700 1 --2- 172.21.15.26:0/3647699222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd20c103cc0 0x7fd20c19b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.172+0000 7fd2113c5700 1 -- 172.21.15.26:0/3647699222 >> 172.21.15.26:0/3647699222 conn(0x7fd20c0fe060 msgr2=0x7fd20c074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:27:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.172+0000 7fd2113c5700 1 -- 172.21.15.26:0/3647699222 shutdown_connections 2024-04-21T10:27:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:27:18.172+0000 7fd2113c5700 1 -- 172.21.15.26:0/3647699222 wait complete. 2024-04-21T10:27:18.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:18 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:18 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:18.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:18 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:27:18.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:18 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:18 smithi026 ceph-mon[27965]: from='osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T10:27:18.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:18 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:27:18.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:18 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:18 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:18 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:18 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:18.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:18 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:27:18.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:18 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:18 smithi190 ceph-mon[31423]: from='osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T10:27:18.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:18 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:27:18.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:18 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:18 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:18 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:18 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:18.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:18 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:27:18.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:18 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:18 smithi169 ceph-mon[31469]: from='osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T10:27:18.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:18 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:27:18.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:18 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:18 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:18.822 DEBUG:teuthology.orchestra.run.smithi026:osd.9> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.9.service 2024-04-21T10:27:18.825 INFO:tasks.cephadm:Deploying osd.10 on smithi169 with /dev/vg_nvme/lv_1... 2024-04-21T10:27:18.825 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T10:27:19.072 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T10:27:19.423 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:19 smithi026 ceph-mon[27965]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:19.424 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:19 smithi026 ceph-mon[27965]: from='osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T10:27:19.424 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:19 smithi026 ceph-mon[27965]: osdmap e67: 10 total, 9 up, 10 in 2024-04-21T10:27:19.424 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:19 smithi026 ceph-mon[27965]: from='osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-04-21T10:27:19.424 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:19 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:19.425 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:27:19 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:27:19.244+0000 7f5b389ab700 -1 osd.9 0 waiting for initial osdmap 2024-04-21T10:27:19.425 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:27:19 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:27:19.260+0000 7f5b3479e700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T10:27:19.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:19 smithi169 ceph-mon[31469]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:19.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:19 smithi169 ceph-mon[31469]: from='osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T10:27:19.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:19 smithi169 ceph-mon[31469]: osdmap e67: 10 total, 9 up, 10 in 2024-04-21T10:27:19.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:19 smithi169 ceph-mon[31469]: from='osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-04-21T10:27:19.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:19 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:19.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:19 smithi190 ceph-mon[31423]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:19.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:19 smithi190 ceph-mon[31423]: from='osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T10:27:19.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:19 smithi190 ceph-mon[31423]: osdmap e67: 10 total, 9 up, 10 in 2024-04-21T10:27:19.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:19 smithi190 ceph-mon[31423]: from='osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-04-21T10:27:19.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:19 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:20.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:20 smithi169 ceph-mon[31469]: from='osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-04-21T10:27:20.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:20 smithi169 ceph-mon[31469]: osdmap e68: 10 total, 9 up, 10 in 2024-04-21T10:27:20.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:20 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:20.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:20 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:20.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:20 smithi190 ceph-mon[31423]: from='osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-04-21T10:27:20.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:20 smithi190 ceph-mon[31423]: osdmap e68: 10 total, 9 up, 10 in 2024-04-21T10:27:20.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:20 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:20.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:20 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:20.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:20 smithi026 ceph-mon[27965]: from='osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-04-21T10:27:20.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:20 smithi026 ceph-mon[27965]: osdmap e68: 10 total, 9 up, 10 in 2024-04-21T10:27:20.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:20 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:20.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:20 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:21.041 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T10:27:21.041 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T10:27:21.041 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-21T10:27:21.041 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-21T10:27:21.041 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115366 s, 909 MB/s 2024-04-21T10:27:21.041 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-21T10:27:21.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:21 smithi026 ceph-mon[27965]: purged_snaps scrub starts 2024-04-21T10:27:21.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:21 smithi026 ceph-mon[27965]: purged_snaps scrub ok 2024-04-21T10:27:21.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:21 smithi026 ceph-mon[27965]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:21.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:21 smithi026 ceph-mon[27965]: osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247] boot 2024-04-21T10:27:21.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:21 smithi026 ceph-mon[27965]: osdmap e69: 10 total, 10 up, 10 in 2024-04-21T10:27:21.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:21 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:21.512 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:21 smithi169 ceph-mon[31469]: purged_snaps scrub starts 2024-04-21T10:27:21.513 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:21 smithi169 ceph-mon[31469]: purged_snaps scrub ok 2024-04-21T10:27:21.513 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:21 smithi169 ceph-mon[31469]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:21.513 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:21 smithi169 ceph-mon[31469]: osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247] boot 2024-04-21T10:27:21.513 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:21 smithi169 ceph-mon[31469]: osdmap e69: 10 total, 10 up, 10 in 2024-04-21T10:27:21.513 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:21 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:21.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:21 smithi190 ceph-mon[31423]: purged_snaps scrub starts 2024-04-21T10:27:21.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:21 smithi190 ceph-mon[31423]: purged_snaps scrub ok 2024-04-21T10:27:21.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:21 smithi190 ceph-mon[31423]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-21T10:27:21.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:21 smithi190 ceph-mon[31423]: osd.9 [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247] boot 2024-04-21T10:27:21.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:21 smithi190 ceph-mon[31423]: osdmap e69: 10 total, 10 up, 10 in 2024-04-21T10:27:21.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:21 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:27:21.720 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_1 2024-04-21T10:27:21.989 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T10:27:22.387 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:22 smithi169 ceph-mon[31469]: osdmap e70: 10 total, 10 up, 10 in 2024-04-21T10:27:22.387 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:22 smithi169 ceph-mon[31469]: Detected new or changed devices on smithi026 2024-04-21T10:27:22.387 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:22 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:22.387 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:22 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:22.387 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:22 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:22.387 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:22 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:22.387 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:22 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:22.387 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:22 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:22.387 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:22 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:22.388 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:22 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:22.388 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:22 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:27:22.388 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:22 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:22.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:22 smithi190 ceph-mon[31423]: osdmap e70: 10 total, 10 up, 10 in 2024-04-21T10:27:22.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:22 smithi190 ceph-mon[31423]: Detected new or changed devices on smithi026 2024-04-21T10:27:22.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:22 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:22.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:22 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:22.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:22 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:22.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:22 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:22.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:22 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:22.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:22 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:22.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:22 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:22.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:22 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:22.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:22 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:27:22.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:22 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:22.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:22 smithi026 ceph-mon[27965]: osdmap e70: 10 total, 10 up, 10 in 2024-04-21T10:27:22.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:22 smithi026 ceph-mon[27965]: Detected new or changed devices on smithi026 2024-04-21T10:27:22.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:22 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:22.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:22 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:22.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:22 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:22.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:22 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:22.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:22 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:22.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:22 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:22.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:22 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:22.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:22 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:22.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:22 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:27:22.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:22 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:23.357 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:23 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi026 to 4273M 2024-04-21T10:27:23.357 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:23 smithi169 ceph-mon[31469]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:23.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:23 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi026 to 4273M 2024-04-21T10:27:23.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:23 smithi190 ceph-mon[31423]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:23.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.538+0000 7f8259e9a700 1 -- 172.21.15.169:0/2742008957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8254100db0 msgr2=0x7f82541011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:27:23.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.538+0000 7f8259e9a700 1 --2- 172.21.15.169:0/2742008957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8254100db0 0x7f82541011d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8244009990 tx=0x7f8244009ca0 comp rx=0 tx=0).stop 2024-04-21T10:27:23.540 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.538+0000 7f8259e9a700 1 -- 172.21.15.169:0/2742008957 shutdown_connections 2024-04-21T10:27:23.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.538+0000 7f8259e9a700 1 --2- 172.21.15.169:0/2742008957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8254100db0 0x7f82541011d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:23.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.538+0000 7f8259e9a700 1 --2- 172.21.15.169:0/2742008957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8254104730 0x7f825410afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:23.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.538+0000 7f8259e9a700 1 --2- 172.21.15.169:0/2742008957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8254103d90 0x7f82541041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:23.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.538+0000 7f8259e9a700 1 -- 172.21.15.169:0/2742008957 >> 172.21.15.169:0/2742008957 conn(0x7f82540fc710 msgr2=0x7f82540feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:27:23.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.538+0000 7f8259e9a700 1 -- 172.21.15.169:0/2742008957 shutdown_connections 2024-04-21T10:27:23.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.538+0000 7f8259e9a700 1 -- 172.21.15.169:0/2742008957 wait complete. 2024-04-21T10:27:23.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.538+0000 7f8259e9a700 1 Processor -- start 2024-04-21T10:27:23.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8259e9a700 1 -- start start 2024-04-21T10:27:23.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8259e9a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8254100db0 0x7f82541036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:27:23.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8259e9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8254103d90 0x7f8254101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:27:23.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8259e9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8254104730 0x7f8254102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:27:23.541 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8259e9a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f825410e550 con 0x7f8254103d90 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8259e9a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f825410e3d0 con 0x7f8254100db0 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8259e9a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f825410e6d0 con 0x7f8254104730 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f82537fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8254100db0 0x7f82541036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f82537fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8254100db0 0x7f82541036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42880/0 (socket says 172.21.15.169:42880) 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f82537fe700 1 -- 172.21.15.169:0/820364090 learned_addr learned my addr 172.21.15.169:0/820364090 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8253fff700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8254104730 0x7f8254102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8252ffd700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8254103d90 0x7f8254101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f82537fe700 1 -- 172.21.15.169:0/820364090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8254104730 msgr2=0x7f8254102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f82537fe700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8254104730 0x7f8254102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f82537fe700 1 -- 172.21.15.169:0/820364090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8254103d90 msgr2=0x7f8254101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f82537fe700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8254103d90 0x7f8254101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f82537fe700 1 -- 172.21.15.169:0/820364090 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8254102b30 con 0x7f8254100db0 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8252ffd700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8254103d90 0x7f8254101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:27:23.542 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f82537fe700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8254100db0 0x7f82541036f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8244006010 tx=0x7f82440057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:27:23.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8250ff9700 1 -- 172.21.15.169:0/820364090 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244027070 con 0x7f8254100db0 2024-04-21T10:27:23.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8250ff9700 1 -- 172.21.15.169:0/820364090 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f824400fc40 con 0x7f8254100db0 2024-04-21T10:27:23.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.539+0000 7f8250ff9700 1 -- 172.21.15.169:0/820364090 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244017650 con 0x7f8254100db0 2024-04-21T10:27:23.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.540+0000 7f8259e9a700 1 -- 172.21.15.169:0/820364090 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82541a2d00 con 0x7f8254100db0 2024-04-21T10:27:23.543 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.540+0000 7f8259e9a700 1 -- 172.21.15.169:0/820364090 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82541a3100 con 0x7f8254100db0 2024-04-21T10:27:23.545 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.541+0000 7f8259e9a700 1 -- 172.21.15.169:0/820364090 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8254069730 con 0x7f8254100db0 2024-04-21T10:27:23.545 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.542+0000 7f8250ff9700 1 -- 172.21.15.169:0/820364090 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f824402a070 con 0x7f8254100db0 2024-04-21T10:27:23.545 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.543+0000 7f8250ff9700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f823c0a3c40 0x7f823c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:27:23.545 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.543+0000 7f8250ff9700 1 -- 172.21.15.169:0/820364090 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7f82440d0fe0 con 0x7f8254100db0 2024-04-21T10:27:23.545 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.543+0000 7f8252ffd700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f823c0a3c40 0x7f823c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:27:23.545 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.543+0000 7f8252ffd700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f823c0a3c40 0x7f823c0a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8248005d90 tx=0x7f8248005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:27:23.547 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.547+0000 7f8250ff9700 1 -- 172.21.15.169:0/820364090 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8244031080 con 0x7f8254100db0 2024-04-21T10:27:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:23 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi026 to 4273M 2024-04-21T10:27:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:23 smithi026 ceph-mon[27965]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:23.754 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:23.754+0000 7f8259e9a700 1 -- 172.21.15.169:0/820364090 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8254062e80 con 0x7f823c0a3c40 2024-04-21T10:27:24.526 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:24 smithi169 ceph-mon[31469]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:27:24.526 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:24 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:27:24.526 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:24 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:27:24.526 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:24 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:24.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:24 smithi190 ceph-mon[31423]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:27:24.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:24 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:27:24.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:24 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:27:24.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:24 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:24.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:24 smithi026 ceph-mon[27965]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:27:24.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:24 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:27:24.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:24 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:27:24.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:24 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:25.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:25 smithi190 ceph-mon[31423]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:25.530 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:25 smithi169 ceph-mon[31469]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:25.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:25 smithi026 ceph-mon[27965]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:27 smithi026 ceph-mon[27965]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:27 smithi026 ceph-mon[27965]: from='client.? 172.21.15.169:0/115484331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11ff534c-dd49-446d-b532-f124b6a70b0b"}]: dispatch 2024-04-21T10:27:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:27 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11ff534c-dd49-446d-b532-f124b6a70b0b"}]: dispatch 2024-04-21T10:27:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:27 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11ff534c-dd49-446d-b532-f124b6a70b0b"}]': finished 2024-04-21T10:27:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:27 smithi026 ceph-mon[27965]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T10:27:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:27 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:27 smithi026 ceph-mon[27965]: from='client.? 172.21.15.169:0/1632316462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:27:27.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:27 smithi169 ceph-mon[31469]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:27.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:27 smithi169 ceph-mon[31469]: from='client.? 172.21.15.169:0/115484331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11ff534c-dd49-446d-b532-f124b6a70b0b"}]: dispatch 2024-04-21T10:27:27.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:27 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11ff534c-dd49-446d-b532-f124b6a70b0b"}]: dispatch 2024-04-21T10:27:27.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:27 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11ff534c-dd49-446d-b532-f124b6a70b0b"}]': finished 2024-04-21T10:27:27.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:27 smithi169 ceph-mon[31469]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T10:27:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:27 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:27.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:27 smithi169 ceph-mon[31469]: from='client.? 172.21.15.169:0/1632316462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:27:27.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:27 smithi190 ceph-mon[31423]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:27.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:27 smithi190 ceph-mon[31423]: from='client.? 172.21.15.169:0/115484331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11ff534c-dd49-446d-b532-f124b6a70b0b"}]: dispatch 2024-04-21T10:27:27.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:27 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11ff534c-dd49-446d-b532-f124b6a70b0b"}]: dispatch 2024-04-21T10:27:27.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:27 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11ff534c-dd49-446d-b532-f124b6a70b0b"}]': finished 2024-04-21T10:27:27.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:27 smithi190 ceph-mon[31423]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T10:27:27.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:27 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:27.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:27 smithi190 ceph-mon[31423]: from='client.? 172.21.15.169:0/1632316462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:27:29.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:29 smithi026 ceph-mon[27965]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:29.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:29 smithi169 ceph-mon[31469]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:29.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:29 smithi190 ceph-mon[31423]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:31.677 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:31 smithi169 ceph-mon[31469]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:31 smithi026 ceph-mon[27965]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:31.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:31 smithi190 ceph-mon[31423]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:33.681 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:33 smithi169 ceph-mon[31469]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:33 smithi026 ceph-mon[27965]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:33.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:33 smithi190 ceph-mon[31423]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:34.549 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:34 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T10:27:34.549 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:34 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:34.549 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:34 smithi169 ceph-mon[31469]: Deploying daemon osd.10 on smithi169 2024-04-21T10:27:34.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T10:27:34.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:34.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:34 smithi026 ceph-mon[27965]: Deploying daemon osd.10 on smithi169 2024-04-21T10:27:34.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:34 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T10:27:34.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:34 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:34.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:34 smithi190 ceph-mon[31423]: Deploying daemon osd.10 on smithi169 2024-04-21T10:27:35.446 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:35 smithi169 ceph-mon[31469]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:35.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:35 smithi026 ceph-mon[27965]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:35.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:35 smithi190 ceph-mon[31423]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:37.551 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:37 smithi169 ceph-mon[31469]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:37.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:37 smithi026 ceph-mon[27965]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:37.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:37 smithi190 ceph-mon[31423]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:39 smithi169 ceph-mon[31469]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:39.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:39 smithi026 ceph-mon[27965]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:39.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:39 smithi190 ceph-mon[31423]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:40.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:40.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:40.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:27:40.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:40.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:40.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:27:40.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:40.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:40.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:27:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:41 smithi026 ceph-mon[27965]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:41.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:41 smithi169 ceph-mon[31469]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:41.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:41 smithi190 ceph-mon[31423]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:43.176 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.177+0000 7f8250ff9700 1 -- 172.21.15.169:0/820364090 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8254062e80 con 0x7f823c0a3c40 2024-04-21T10:27:43.177 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 10 on host 'smithi169' 2024-04-21T10:27:43.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.179+0000 7f8259e9a700 1 -- 172.21.15.169:0/820364090 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f823c0a3c40 msgr2=0x7f823c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:27:43.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.179+0000 7f8259e9a700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f823c0a3c40 0x7f823c0a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8248005d90 tx=0x7f8248005d20 comp rx=0 tx=0).stop 2024-04-21T10:27:43.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.179+0000 7f8259e9a700 1 -- 172.21.15.169:0/820364090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8254100db0 msgr2=0x7f82541036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:27:43.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.179+0000 7f8259e9a700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8254100db0 0x7f82541036f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8244006010 tx=0x7f82440057a0 comp rx=0 tx=0).stop 2024-04-21T10:27:43.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.179+0000 7f8259e9a700 1 -- 172.21.15.169:0/820364090 shutdown_connections 2024-04-21T10:27:43.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.179+0000 7f8259e9a700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f823c0a3c40 0x7f823c0a6100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:43.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.179+0000 7f8259e9a700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8254100db0 0x7f82541036f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:43.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.179+0000 7f8259e9a700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8254104730 0x7f8254102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:43.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.179+0000 7f8259e9a700 1 --2- 172.21.15.169:0/820364090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8254103d90 0x7f8254101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:43.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.179+0000 7f8259e9a700 1 -- 172.21.15.169:0/820364090 >> 172.21.15.169:0/820364090 conn(0x7f82540fc710 msgr2=0x7f8254107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:27:43.180 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.180+0000 7f8259e9a700 1 -- 172.21.15.169:0/820364090 shutdown_connections 2024-04-21T10:27:43.180 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-21T10:27:43.180+0000 7f8259e9a700 1 -- 172.21.15.169:0/820364090 wait complete. 2024-04-21T10:27:43.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:43 smithi169 ceph-mon[31469]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:43.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:43.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:27:43.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:27:43.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:43 smithi169 ceph-mon[31469]: from='osd.10 [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T10:27:43.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:43 smithi169 ceph-mon[31469]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T10:27:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:43 smithi190 ceph-mon[31423]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:27:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:27:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:43 smithi190 ceph-mon[31423]: from='osd.10 [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T10:27:43.593 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:43 smithi190 ceph-mon[31423]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T10:27:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:43 smithi026 ceph-mon[27965]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:27:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:43.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:27:43.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:43 smithi026 ceph-mon[27965]: from='osd.10 [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T10:27:43.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:43 smithi026 ceph-mon[27965]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T10:27:43.839 DEBUG:teuthology.orchestra.run.smithi169:osd.10> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.10.service 2024-04-21T10:27:43.841 INFO:tasks.cephadm:Deploying osd.11 on smithi190 with /dev/vg_nvme/lv_1... 2024-04-21T10:27:43.841 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T10:27:44.087 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:27:45.291 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:45 smithi169 ceph-mon[31469]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T10:27:45.291 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:45 smithi169 ceph-mon[31469]: osdmap e72: 11 total, 10 up, 11 in 2024-04-21T10:27:45.291 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:45 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:45.291 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:45 smithi169 ceph-mon[31469]: from='osd.10 [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:27:45.291 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:45 smithi169 ceph-mon[31469]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:27:45.291 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:45 smithi169 ceph-mon[31469]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:45.291 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:27:45 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:27:45.165+0000 7f3d0c520700 -1 osd.10 0 waiting for initial osdmap 2024-04-21T10:27:45.292 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:27:45 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:27:45.180+0000 7f3d08313700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T10:27:45.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:45 smithi026 ceph-mon[27965]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T10:27:45.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:45 smithi026 ceph-mon[27965]: osdmap e72: 11 total, 10 up, 11 in 2024-04-21T10:27:45.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:45 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:45.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:45 smithi026 ceph-mon[27965]: from='osd.10 [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:27:45.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:45 smithi026 ceph-mon[27965]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:27:45.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:45 smithi026 ceph-mon[27965]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:45.480 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:45 smithi190 ceph-mon[31423]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T10:27:45.480 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:45 smithi190 ceph-mon[31423]: osdmap e72: 11 total, 10 up, 11 in 2024-04-21T10:27:45.480 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:45 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:45.480 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:45 smithi190 ceph-mon[31423]: from='osd.10 [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:27:45.480 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:45 smithi190 ceph-mon[31423]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-21T10:27:45.480 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:45 smithi190 ceph-mon[31423]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:46.226 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T10:27:46.226 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T10:27:46.226 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-04-21T10:27:46.226 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-04-21T10:27:46.226 INFO:teuthology.orchestra.run.smithi190.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113768 s, 922 MB/s 2024-04-21T10:27:46.226 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 2024-04-21T10:27:46.226 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-04-21T10:27:46.417 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:46 smithi169 ceph-mon[31469]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-21T10:27:46.417 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:46 smithi169 ceph-mon[31469]: osdmap e73: 11 total, 10 up, 11 in 2024-04-21T10:27:46.417 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:46 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:46.417 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:46 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:46.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:46 smithi026 ceph-mon[27965]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-21T10:27:46.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:46 smithi026 ceph-mon[27965]: osdmap e73: 11 total, 10 up, 11 in 2024-04-21T10:27:46.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:46 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:46.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:46 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:46.525 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:46 smithi190 ceph-mon[31423]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-21T10:27:46.525 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:46 smithi190 ceph-mon[31423]: osdmap e73: 11 total, 10 up, 11 in 2024-04-21T10:27:46.525 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:46 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:46.525 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:46 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:46.813 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi190:vg_nvme/lv_1 2024-04-21T10:27:47.050 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:27:47.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: purged_snaps scrub starts 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: purged_snaps scrub ok 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: osd.10 [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945] boot 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: Detected new or changed devices on smithi169 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:47.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-21T10:27:47.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:47.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:47.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:27:47.437 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:47 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:47.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: purged_snaps scrub starts 2024-04-21T10:27:47.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: purged_snaps scrub ok 2024-04-21T10:27:47.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:47.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:47.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: osd.10 [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945] boot 2024-04-21T10:27:47.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T10:27:47.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:47.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: Detected new or changed devices on smithi169 2024-04-21T10:27:47.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:47.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:47.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:47.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:47.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:47.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:47.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-21T10:27:47.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:47.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:47.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:27:47.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:47 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:47.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: purged_snaps scrub starts 2024-04-21T10:27:47.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: purged_snaps scrub ok 2024-04-21T10:27:47.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-21T10:27:47.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:47.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: osd.10 [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945] boot 2024-04-21T10:27:47.495 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: Detected new or changed devices on smithi169 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:27:47.496 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:47 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:27:48.240 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:48 smithi190 ceph-mon[31423]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T10:27:48.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:48 smithi169 ceph-mon[31469]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T10:27:48.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:48 smithi026 ceph-mon[27965]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T10:27:48.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.721+0000 7f2c92c4f700 1 -- 172.21.15.190:0/3991275961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c8c102b90 msgr2=0x7f2c8c102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:27:48.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.721+0000 7f2c92c4f700 1 --2- 172.21.15.190:0/3991275961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c8c102b90 0x7f2c8c102ff0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009990 tx=0x7f2c78009ca0 comp rx=0 tx=0).stop 2024-04-21T10:27:48.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.721+0000 7f2c92c4f700 1 -- 172.21.15.190:0/3991275961 shutdown_connections 2024-04-21T10:27:48.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.721+0000 7f2c92c4f700 1 --2- 172.21.15.190:0/3991275961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c8c103530 0x7f2c8c109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:48.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.721+0000 7f2c92c4f700 1 --2- 172.21.15.190:0/3991275961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c8c101990 0x7f2c8c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:48.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.721+0000 7f2c92c4f700 1 --2- 172.21.15.190:0/3991275961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c8c102b90 0x7f2c8c102ff0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:48.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.721+0000 7f2c92c4f700 1 -- 172.21.15.190:0/3991275961 >> 172.21.15.190:0/3991275961 conn(0x7f2c8c0fcf30 msgr2=0x7f2c8c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:27:48.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.721+0000 7f2c92c4f700 1 -- 172.21.15.190:0/3991275961 shutdown_connections 2024-04-21T10:27:48.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.721+0000 7f2c92c4f700 1 -- 172.21.15.190:0/3991275961 wait complete. 2024-04-21T10:27:48.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c92c4f700 1 Processor -- start 2024-04-21T10:27:48.723 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c92c4f700 1 -- start start 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c92c4f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c8c101990 0x7f2c8c105920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c92c4f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c8c102b90 0x7f2c8c103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c92c4f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c8c103530 0x7f2c8c1044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c92c4f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c10d480 con 0x7f2c8c101990 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c92c4f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c10d300 con 0x7f2c8c103530 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c92c4f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c10d600 con 0x7f2c8c102b90 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c8bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c8c102b90 0x7f2c8c103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c8bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c8c102b90 0x7f2c8c103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:58476/0 (socket says 172.21.15.190:58476) 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c8bfff700 1 -- 172.21.15.190:0/887838488 learned_addr learned my addr 172.21.15.190:0/887838488 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c911ec700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c8c103530 0x7f2c8c1044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.722+0000 7f2c909eb700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c8c101990 0x7f2c8c105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.723+0000 7f2c8bfff700 1 -- 172.21.15.190:0/887838488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c8c103530 msgr2=0x7f2c8c1044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:27:48.724 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.723+0000 7f2c8bfff700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c8c103530 0x7f2c8c1044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:48.725 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.723+0000 7f2c8bfff700 1 -- 172.21.15.190:0/887838488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c8c101990 msgr2=0x7f2c8c105920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:27:48.725 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.723+0000 7f2c8bfff700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c8c101990 0x7f2c8c105920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:27:48.725 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.723+0000 7f2c8bfff700 1 -- 172.21.15.190:0/887838488 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c104d60 con 0x7f2c8c102b90 2024-04-21T10:27:48.725 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.723+0000 7f2c909eb700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c8c101990 0x7f2c8c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:27:48.725 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.723+0000 7f2c8bfff700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c8c102b90 0x7f2c8c103f70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2c78006010 tx=0x7f2c78009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:27:48.725 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.723+0000 7f2c89ffb700 1 -- 172.21.15.190:0/887838488 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78027070 con 0x7f2c8c102b90 2024-04-21T10:27:48.725 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.723+0000 7f2c92c4f700 1 -- 172.21.15.190:0/887838488 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8c1a2d60 con 0x7f2c8c102b90 2024-04-21T10:27:48.725 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.723+0000 7f2c89ffb700 1 -- 172.21.15.190:0/887838488 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c78022470 con 0x7f2c8c102b90 2024-04-21T10:27:48.725 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.723+0000 7f2c89ffb700 1 -- 172.21.15.190:0/887838488 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7800f6e0 con 0x7f2c8c102b90 2024-04-21T10:27:48.725 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.723+0000 7f2c92c4f700 1 -- 172.21.15.190:0/887838488 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8c1a31b0 con 0x7f2c8c102b90 2024-04-21T10:27:48.726 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.724+0000 7f2c92c4f700 1 -- 172.21.15.190:0/887838488 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c8c1a3380 con 0x7f2c8c102b90 2024-04-21T10:27:48.726 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.725+0000 7f2c89ffb700 1 -- 172.21.15.190:0/887838488 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2c7802a070 con 0x7f2c8c102b90 2024-04-21T10:27:48.726 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.726+0000 7f2c89ffb700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f2c7c0a3c40 0x7f2c7c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:27:48.727 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.726+0000 7f2c89ffb700 1 -- 172.21.15.190:0/887838488 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6723+0+0 (secure 0 0 0) 0x7f2c780d0dc0 con 0x7f2c8c102b90 2024-04-21T10:27:48.727 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.726+0000 7f2c909eb700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f2c7c0a3c40 0x7f2c7c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:27:48.727 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.726+0000 7f2c909eb700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f2c7c0a3c40 0x7f2c7c0a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2c80005950 tx=0x7f2c800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:27:48.729 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.730+0000 7f2c89ffb700 1 -- 172.21.15.190:0/887838488 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c78013070 con 0x7f2c8c102b90 2024-04-21T10:27:48.936 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:27:48.937+0000 7f2c92c4f700 1 -- 172.21.15.190:0/887838488 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2c8c06b6a0 con 0x7f2c7c0a3c40 2024-04-21T10:27:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:49 smithi169 ceph-mon[31469]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:27:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:49 smithi169 ceph-mon[31469]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T10:27:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:49 smithi169 ceph-mon[31469]: osdmap e77: 11 total, 11 up, 11 in 2024-04-21T10:27:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:49 smithi169 ceph-mon[31469]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:27:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:27:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:27:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:49 smithi190 ceph-mon[31423]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:27:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:49 smithi190 ceph-mon[31423]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T10:27:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:49 smithi190 ceph-mon[31423]: osdmap e77: 11 total, 11 up, 11 in 2024-04-21T10:27:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:49 smithi190 ceph-mon[31423]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:27:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:27:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:27:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:49 smithi026 ceph-mon[27965]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:27:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:49 smithi026 ceph-mon[27965]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T10:27:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:49 smithi026 ceph-mon[27965]: osdmap e77: 11 total, 11 up, 11 in 2024-04-21T10:27:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:49 smithi026 ceph-mon[27965]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:27:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T10:27:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T10:27:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:27:50.817 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:50 smithi190 ceph-mon[31423]: osdmap e78: 11 total, 11 up, 11 in 2024-04-21T10:27:51.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:50 smithi026 ceph-mon[27965]: osdmap e78: 11 total, 11 up, 11 in 2024-04-21T10:27:51.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:50 smithi169 ceph-mon[31469]: osdmap e78: 11 total, 11 up, 11 in 2024-04-21T10:27:52.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:51 smithi190 ceph-mon[31423]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:27:52.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:51 smithi190 ceph-mon[31423]: from='client.? 172.21.15.190:0/1588294439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef"}]: dispatch 2024-04-21T10:27:52.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:51 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef"}]: dispatch 2024-04-21T10:27:52.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:51 smithi190 ceph-mon[31423]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef"}]': finished 2024-04-21T10:27:52.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:51 smithi190 ceph-mon[31423]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T10:27:52.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:27:52.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:51 smithi026 ceph-mon[27965]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:27:52.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:51 smithi026 ceph-mon[27965]: from='client.? 172.21.15.190:0/1588294439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef"}]: dispatch 2024-04-21T10:27:52.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:51 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef"}]: dispatch 2024-04-21T10:27:52.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:51 smithi026 ceph-mon[27965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef"}]': finished 2024-04-21T10:27:52.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:51 smithi026 ceph-mon[27965]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T10:27:52.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:27:52.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:51 smithi169 ceph-mon[31469]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:27:52.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:51 smithi169 ceph-mon[31469]: from='client.? 172.21.15.190:0/1588294439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef"}]: dispatch 2024-04-21T10:27:52.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:51 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef"}]: dispatch 2024-04-21T10:27:52.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:51 smithi169 ceph-mon[31469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef"}]': finished 2024-04-21T10:27:52.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:51 smithi169 ceph-mon[31469]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T10:27:52.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:27:53.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:52 smithi026 ceph-mon[27965]: from='client.? 172.21.15.190:0/339081064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:27:53.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:52 smithi169 ceph-mon[31469]: from='client.? 172.21.15.190:0/339081064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:27:53.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:52 smithi190 ceph-mon[31423]: from='client.? 172.21.15.190:0/339081064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T10:27:54.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:53 smithi026 ceph-mon[27965]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:27:54.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:53 smithi169 ceph-mon[31469]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:27:54.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:53 smithi190 ceph-mon[31423]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T10:27:55.998 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:55 smithi190 ceph-mon[31423]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-21T10:27:56.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:55 smithi026 ceph-mon[27965]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-21T10:27:56.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:55 smithi169 ceph-mon[31469]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-21T10:27:58.094 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:57 smithi190 ceph-mon[31423]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-21T10:27:58.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:57 smithi026 ceph-mon[27965]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-21T10:27:58.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:57 smithi169 ceph-mon[31469]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-21T10:28:00.114 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:59 smithi190 ceph-mon[31423]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-21T10:28:00.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T10:28:00.115 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:27:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:00.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:59 smithi026 ceph-mon[27965]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-21T10:28:00.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T10:28:00.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:27:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:00.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:59 smithi169 ceph-mon[31469]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-21T10:28:00.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:59 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T10:28:00.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:27:59 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:01.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:00 smithi190 ceph-mon[31423]: Deploying daemon osd.11 on smithi190 2024-04-21T10:28:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:00 smithi026 ceph-mon[27965]: Deploying daemon osd.11 on smithi190 2024-04-21T10:28:01.197 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:00 smithi169 ceph-mon[31469]: Deploying daemon osd.11 on smithi190 2024-04-21T10:28:02.164 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:01 smithi190 ceph-mon[31423]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T10:28:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:01 smithi026 ceph-mon[27965]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T10:28:02.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:01 smithi169 ceph-mon[31469]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T10:28:04.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:03 smithi190 ceph-mon[31423]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:03 smithi026 ceph-mon[27965]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:03 smithi169 ceph-mon[31469]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:05.758 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:05 smithi190 ceph-mon[31423]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:05.759 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:05 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:05.759 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:05 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:05.759 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:05 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:05.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:05 smithi026 ceph-mon[27965]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:05.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:05.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:05.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:05 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:05 smithi169 ceph-mon[31469]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:05 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:05 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:05 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:07.702 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.703+0000 7f2c89ffb700 1 -- 172.21.15.190:0/887838488 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2c8c06b6a0 con 0x7f2c7c0a3c40 2024-04-21T10:28:07.703 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 11 on host 'smithi190' 2024-04-21T10:28:07.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.705+0000 7f2c92c4f700 1 -- 172.21.15.190:0/887838488 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f2c7c0a3c40 msgr2=0x7f2c7c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:07.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.705+0000 7f2c92c4f700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f2c7c0a3c40 0x7f2c7c0a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2c80005950 tx=0x7f2c800058e0 comp rx=0 tx=0).stop 2024-04-21T10:28:07.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.705+0000 7f2c92c4f700 1 -- 172.21.15.190:0/887838488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c8c102b90 msgr2=0x7f2c8c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:07.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.705+0000 7f2c92c4f700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c8c102b90 0x7f2c8c103f70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2c78006010 tx=0x7f2c78009f90 comp rx=0 tx=0).stop 2024-04-21T10:28:07.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.705+0000 7f2c92c4f700 1 -- 172.21.15.190:0/887838488 shutdown_connections 2024-04-21T10:28:07.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.705+0000 7f2c92c4f700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f2c7c0a3c40 0x7f2c7c0a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:07.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.705+0000 7f2c92c4f700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c8c103530 0x7f2c8c1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:07.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.705+0000 7f2c92c4f700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c8c101990 0x7f2c8c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:07.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.705+0000 7f2c92c4f700 1 --2- 172.21.15.190:0/887838488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c8c102b90 0x7f2c8c103f70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:07.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.705+0000 7f2c92c4f700 1 -- 172.21.15.190:0/887838488 >> 172.21.15.190:0/887838488 conn(0x7f2c8c0fcf30 msgr2=0x7f2c8c06af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:07.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.705+0000 7f2c92c4f700 1 -- 172.21.15.190:0/887838488 shutdown_connections 2024-04-21T10:28:07.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:07.705+0000 7f2c92c4f700 1 -- 172.21.15.190:0/887838488 wait complete. 2024-04-21T10:28:07.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:07 smithi026 ceph-mon[27965]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:07.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:07 smithi169 ceph-mon[31469]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:08.006 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:07 smithi190 ceph-mon[31423]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:08.335 DEBUG:teuthology.orchestra.run.smithi190:osd.11> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.11.service 2024-04-21T10:28:08.338 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-21T10:28:08.338 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-21T10:28:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:28:08.674 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:28:08 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:28:08.486+0000 7f81fe92b7c0 -1 osd.11 0 log_to_monitors true 2024-04-21T10:28:08.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:08 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:08.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:08 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:08.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:08 smithi026 ceph-mon[27965]: from='osd.11 [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T10:28:08.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:08 smithi026 ceph-mon[27965]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T10:28:08.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:08 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:08.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:08 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:08.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:08 smithi169 ceph-mon[31469]: from='osd.11 [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T10:28:08.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:08 smithi169 ceph-mon[31469]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T10:28:09.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:08 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:09.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:08 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:09.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:08 smithi190 ceph-mon[31423]: from='osd.11 [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T10:28:09.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:08 smithi190 ceph-mon[31423]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T10:28:09.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:09 smithi190 ceph-mon[31423]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:09.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:09 smithi190 ceph-mon[31423]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T10:28:09.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:09 smithi190 ceph-mon[31423]: osdmap e80: 12 total, 11 up, 12 in 2024-04-21T10:28:09.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:09.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:09 smithi190 ceph-mon[31423]: from='osd.11 [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:28:09.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:09 smithi190 ceph-mon[31423]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:28:09.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:09.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:09 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:09.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:09 smithi169 ceph-mon[31469]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:09.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:09 smithi169 ceph-mon[31469]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T10:28:09.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:09 smithi169 ceph-mon[31469]: osdmap e80: 12 total, 11 up, 12 in 2024-04-21T10:28:09.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:09.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:09 smithi169 ceph-mon[31469]: from='osd.11 [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:28:09.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:09 smithi169 ceph-mon[31469]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:28:09.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:09.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:09 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:10.064 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:09 smithi026 ceph-mon[27965]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:10.064 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:09 smithi026 ceph-mon[27965]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T10:28:10.065 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:09 smithi026 ceph-mon[27965]: osdmap e80: 12 total, 11 up, 12 in 2024-04-21T10:28:10.065 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:10.065 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:09 smithi026 ceph-mon[27965]: from='osd.11 [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:28:10.065 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:09 smithi026 ceph-mon[27965]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-04-21T10:28:10.065 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:10.065 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:09 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.393+0000 7f4e6fefb700 1 -- 172.21.15.26:0/161711188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e68105e60 msgr2=0x7f4e68108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.393+0000 7f4e6fefb700 1 --2- 172.21.15.26:0/161711188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e68105e60 0x7f4e68108250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009990 tx=0x7f4e5c009ca0 comp rx=0 tx=0).stop 2024-04-21T10:28:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.393+0000 7f4e6fefb700 1 -- 172.21.15.26:0/161711188 shutdown_connections 2024-04-21T10:28:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.393+0000 7f4e6fefb700 1 --2- 172.21.15.26:0/161711188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e68103530 0x7f4e68105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.393+0000 7f4e6fefb700 1 --2- 172.21.15.26:0/161711188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e68108790 0x7f4e6810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.393+0000 7f4e6fefb700 1 --2- 172.21.15.26:0/161711188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e68105e60 0x7f4e68108250 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.393+0000 7f4e6fefb700 1 -- 172.21.15.26:0/161711188 >> 172.21.15.26:0/161711188 conn(0x7f4e680fcf40 msgr2=0x7f4e680ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.393+0000 7f4e6fefb700 1 -- 172.21.15.26:0/161711188 shutdown_connections 2024-04-21T10:28:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.393+0000 7f4e6fefb700 1 -- 172.21.15.26:0/161711188 wait complete. 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6fefb700 1 Processor -- start 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6fefb700 1 -- start start 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6fefb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e68103530 0x7f4e68198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6fefb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e68105e60 0x7f4e68199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6fefb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e68108790 0x7f4e681a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6fefb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6810e550 con 0x7f4e68103530 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6fefb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6810e3d0 con 0x7f4e68108790 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6fefb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6810e6d0 con 0x7f4e68105e60 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6dc97700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e68103530 0x7f4e68198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6d496700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e68105e60 0x7f4e68199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6dc97700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e68103530 0x7f4e68198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54526/0 (socket says 172.21.15.26:54526) 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6d496700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e68105e60 0x7f4e68199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.26:50034/0 (socket says 172.21.15.26:50034) 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6dc97700 1 -- 172.21.15.26:0/1048857654 learned_addr learned my addr 172.21.15.26:0/1048857654 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6dc97700 1 -- 172.21.15.26:0/1048857654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e68105e60 msgr2=0x7f4e68199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6e498700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e68108790 0x7f4e681a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6dc97700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e68105e60 0x7f4e68199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6dc97700 1 -- 172.21.15.26:0/1048857654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e68108790 msgr2=0x7f4e681a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6dc97700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e68108790 0x7f4e681a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6dc97700 1 -- 172.21.15.26:0/1048857654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e681a08a0 con 0x7f4e68103530 2024-04-21T10:28:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6d496700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e68105e60 0x7f4e68199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:28:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.394+0000 7f4e6dc97700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e68103530 0x7f4e68198c50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e580060b0 tx=0x7f4e5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.395+0000 7f4e66ffd700 1 -- 172.21.15.26:0/1048857654 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5801a070 con 0x7f4e68103530 2024-04-21T10:28:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.395+0000 7f4e66ffd700 1 -- 172.21.15.26:0/1048857654 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e5800ddd0 con 0x7f4e68103530 2024-04-21T10:28:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.395+0000 7f4e6fefb700 1 -- 172.21.15.26:0/1048857654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e681a0b90 con 0x7f4e68103530 2024-04-21T10:28:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.395+0000 7f4e66ffd700 1 -- 172.21.15.26:0/1048857654 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58014720 con 0x7f4e68103530 2024-04-21T10:28:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.395+0000 7f4e6fefb700 1 -- 172.21.15.26:0/1048857654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e681a1120 con 0x7f4e68103530 2024-04-21T10:28:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.396+0000 7f4e6fefb700 1 -- 172.21.15.26:0/1048857654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e6802df80 con 0x7f4e68103530 2024-04-21T10:28:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.397+0000 7f4e66ffd700 1 -- 172.21.15.26:0/1048857654 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4e5801d070 con 0x7f4e68103530 2024-04-21T10:28:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.398+0000 7f4e66ffd700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4e500a3cc0 0x7f4e500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.398+0000 7f4e6d496700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4e500a3cc0 0x7f4e500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.398+0000 7f4e66ffd700 1 -- 172.21.15.26:0/1048857654 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6847+0+0 (secure 0 0 0) 0x7f4e580cf610 con 0x7f4e68103530 2024-04-21T10:28:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.398+0000 7f4e6d496700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4e500a3cc0 0x7f4e500a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00b760 tx=0x7f4e5c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.402+0000 7f4e66ffd700 1 -- 172.21.15.26:0/1048857654 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e58010070 con 0x7f4e68103530 2024-04-21T10:28:10.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.610+0000 7f4e6fefb700 1 -- 172.21.15.26:0/1048857654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4e6802e380 con 0x7f4e68103530 2024-04-21T10:28:10.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.610+0000 7f4e66ffd700 1 -- 172.21.15.26:0/1048857654 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f4e5809c5b0 con 0x7f4e68103530 2024-04-21T10:28:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:10.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.613+0000 7f4e6fefb700 1 -- 172.21.15.26:0/1048857654 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4e500a3cc0 msgr2=0x7f4e500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:10.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.613+0000 7f4e6fefb700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4e500a3cc0 0x7f4e500a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00b760 tx=0x7f4e5c01a040 comp rx=0 tx=0).stop 2024-04-21T10:28:10.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.613+0000 7f4e6fefb700 1 -- 172.21.15.26:0/1048857654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e68103530 msgr2=0x7f4e68198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:10.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.613+0000 7f4e6fefb700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e68103530 0x7f4e68198c50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e580060b0 tx=0x7f4e5800d3b0 comp rx=0 tx=0).stop 2024-04-21T10:28:10.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.613+0000 7f4e6fefb700 1 -- 172.21.15.26:0/1048857654 shutdown_connections 2024-04-21T10:28:10.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.613+0000 7f4e6fefb700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4e500a3cc0 0x7f4e500a6180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:10.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.613+0000 7f4e6fefb700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e68108790 0x7f4e681a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:10.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.613+0000 7f4e6fefb700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e68103530 0x7f4e68198c50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:10.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.613+0000 7f4e6fefb700 1 --2- 172.21.15.26:0/1048857654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e68105e60 0x7f4e68199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:10.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.613+0000 7f4e6fefb700 1 -- 172.21.15.26:0/1048857654 >> 172.21.15.26:0/1048857654 conn(0x7f4e680fcf40 msgr2=0x7f4e68103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:10.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.613+0000 7f4e6fefb700 1 -- 172.21.15.26:0/1048857654 shutdown_connections 2024-04-21T10:28:10.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:10.613+0000 7f4e6fefb700 1 -- 172.21.15.26:0/1048857654 wait complete. 2024-04-21T10:28:10.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:28:10 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:28:10.580+0000 7f81f4791700 -1 osd.11 0 waiting for initial osdmap 2024-04-21T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:10 smithi190 ceph-mon[31423]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-04-21T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:10 smithi190 ceph-mon[31423]: osdmap e81: 12 total, 11 up, 12 in 2024-04-21T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:10 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:10 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:10 smithi190 ceph-mon[31423]: from='osd.11 ' entity='osd.11' 2024-04-21T10:28:10.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:10 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1048857654' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T10:28:10.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:10 smithi169 ceph-mon[31469]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-04-21T10:28:10.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:10 smithi169 ceph-mon[31469]: osdmap e81: 12 total, 11 up, 12 in 2024-04-21T10:28:10.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:10 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:10.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:10 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:10.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:10 smithi169 ceph-mon[31469]: from='osd.11 ' entity='osd.11' 2024-04-21T10:28:10.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:10 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1048857654' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T10:28:11.172 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:10 smithi026 ceph-mon[27965]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-04-21T10:28:11.172 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:10 smithi026 ceph-mon[27965]: osdmap e81: 12 total, 11 up, 12 in 2024-04-21T10:28:11.172 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:10 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:11.172 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:10 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:11.172 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:10 smithi026 ceph-mon[27965]: from='osd.11 ' entity='osd.11' 2024-04-21T10:28:11.172 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:10 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1048857654' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T10:28:11.201 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"num_osds":12,"num_up_osds":11,"osd_up_since":1713695266,"num_in_osds":12,"osd_in_since":1713695271,"num_remapped_pgs":0} 2024-04-21T10:28:11.276 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:28:10 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:28:10.777+0000 7f81ebd7b700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T10:28:11.921 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:11 smithi190 ceph-mon[31423]: purged_snaps scrub starts 2024-04-21T10:28:11.921 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:11 smithi190 ceph-mon[31423]: purged_snaps scrub ok 2024-04-21T10:28:11.922 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:11 smithi190 ceph-mon[31423]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:11.922 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:11 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:11.922 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:11 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:11.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:11 smithi026 ceph-mon[27965]: purged_snaps scrub starts 2024-04-21T10:28:11.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:11 smithi026 ceph-mon[27965]: purged_snaps scrub ok 2024-04-21T10:28:11.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:11 smithi026 ceph-mon[27965]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:11.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:11 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:11.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:11 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:12.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:11 smithi169 ceph-mon[31469]: purged_snaps scrub starts 2024-04-21T10:28:12.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:11 smithi169 ceph-mon[31469]: purged_snaps scrub ok 2024-04-21T10:28:12.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:11 smithi169 ceph-mon[31469]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:12.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:11 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:12.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:11 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:12.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-21T10:28:12.455 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:28:12.737 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: osd.11 [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638] boot 2024-04-21T10:28:12.737 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T10:28:12.737 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:12.737 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:12.738 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:12.738 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:28:12.738 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:28:12.738 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:28:12.738 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:28:12.738 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:12.738 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:12.738 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:28:12.738 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:12.738 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:12.738 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:12.738 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:28:12.739 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:12 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:13.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: osd.11 [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638] boot 2024-04-21T10:28:13.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T10:28:13.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:13.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:13.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:13.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:28:13.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:28:13.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:28:13.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:28:13.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:13.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:13.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:28:13.029 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:13.029 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:13.029 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:13.029 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:28:13.029 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:12 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:13.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: osd.11 [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638] boot 2024-04-21T10:28:13.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T10:28:13.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:28:13.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:13.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:13.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:28:13.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:28:13.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:28:13.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:28:13.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:13.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:13.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:28:13.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:13.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:13.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:13.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:28:13.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:12 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:13 smithi026 ceph-mon[27965]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:13 smithi026 ceph-mon[27965]: Detected new or changed devices on smithi190 2024-04-21T10:28:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:13 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi190 to 4272M 2024-04-21T10:28:13.884 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:13 smithi026 ceph-mon[27965]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T10:28:14.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:13 smithi190 ceph-mon[31423]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:14.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:13 smithi190 ceph-mon[31423]: Detected new or changed devices on smithi190 2024-04-21T10:28:14.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:13 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi190 to 4272M 2024-04-21T10:28:14.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:13 smithi190 ceph-mon[31423]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T10:28:14.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:13 smithi169 ceph-mon[31469]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-21T10:28:14.099 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:13 smithi169 ceph-mon[31469]: Detected new or changed devices on smithi190 2024-04-21T10:28:14.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:13 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi190 to 4272M 2024-04-21T10:28:14.100 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:13 smithi169 ceph-mon[31469]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T10:28:14.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.218+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/3752124464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a0103cb0 msgr2=0x7fd0a0104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:14.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.218+0000 7fd0a7f0a700 1 --2- 172.21.15.26:0/3752124464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a0103cb0 0x7fd0a0104110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd0940099c0 tx=0x7fd094009cd0 comp rx=0 tx=0).stop 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.218+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/3752124464 shutdown_connections 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.218+0000 7fd0a7f0a700 1 --2- 172.21.15.26:0/3752124464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0a0102ab0 0x7fd0a0102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.218+0000 7fd0a7f0a700 1 --2- 172.21.15.26:0/3752124464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a0104650 0x7fd0a010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.218+0000 7fd0a7f0a700 1 --2- 172.21.15.26:0/3752124464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a0103cb0 0x7fd0a0104110 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.218+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/3752124464 >> 172.21.15.26:0/3752124464 conn(0x7fd0a00fe010 msgr2=0x7fd0a0100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.218+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/3752124464 shutdown_connections 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.218+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/3752124464 wait complete. 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.219+0000 7fd0a7f0a700 1 Processor -- start 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.219+0000 7fd0a7f0a700 1 -- start start 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.219+0000 7fd0a7f0a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a0102ab0 0x7fd0a00767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.219+0000 7fd0a7f0a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a0103cb0 0x7fd0a0074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.219+0000 7fd0a7f0a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0a0104650 0x7fd0a0075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.219+0000 7fd0a7f0a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a010e5e0 con 0x7fd0a0103cb0 2024-04-21T10:28:14.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.219+0000 7fd0a7f0a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a010e460 con 0x7fd0a0104650 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.219+0000 7fd0a7f0a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a010e760 con 0x7fd0a0102ab0 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a54a5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a0103cb0 0x7fd0a0074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a54a5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a0103cb0 0x7fd0a0074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54542/0 (socket says 172.21.15.26:54542) 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a54a5700 1 -- 172.21.15.26:0/1056011722 learned_addr learned my addr 172.21.15.26:0/1056011722 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a5ca6700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a0102ab0 0x7fd0a00767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a64a7700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0a0104650 0x7fd0a0075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a54a5700 1 -- 172.21.15.26:0/1056011722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a0102ab0 msgr2=0x7fd0a00767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a54a5700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a0102ab0 0x7fd0a00767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a54a5700 1 -- 172.21.15.26:0/1056011722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0a0104650 msgr2=0x7fd0a0075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a54a5700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0a0104650 0x7fd0a0075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a54a5700 1 -- 172.21.15.26:0/1056011722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a0075c20 con 0x7fd0a0103cb0 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a64a7700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0a0104650 0x7fd0a0075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:28:14.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a54a5700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a0103cb0 0x7fd0a0074e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd094003790 tx=0x7fd094005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:14.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd092ffd700 1 -- 172.21.15.26:0/1056011722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094027070 con 0x7fd0a0103cb0 2024-04-21T10:28:14.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/1056011722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a01a2d80 con 0x7fd0a0103cb0 2024-04-21T10:28:14.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.220+0000 7fd092ffd700 1 -- 172.21.15.26:0/1056011722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd094004160 con 0x7fd0a0103cb0 2024-04-21T10:28:14.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.221+0000 7fd092ffd700 1 -- 172.21.15.26:0/1056011722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094022d90 con 0x7fd0a0103cb0 2024-04-21T10:28:14.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.221+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/1056011722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a01a3200 con 0x7fd0a0103cb0 2024-04-21T10:28:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.222+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/1056011722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a0069730 con 0x7fd0a0103cb0 2024-04-21T10:28:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.222+0000 7fd092ffd700 1 -- 172.21.15.26:0/1056011722 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd09402a070 con 0x7fd0a0103cb0 2024-04-21T10:28:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.223+0000 7fd092ffd700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd08c0a3c40 0x7fd08c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.223+0000 7fd0a5ca6700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd08c0a3c40 0x7fd08c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.223+0000 7fd092ffd700 1 -- 172.21.15.26:0/1056011722 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fd0940d0dd0 con 0x7fd0a0103cb0 2024-04-21T10:28:14.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.223+0000 7fd0a5ca6700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd08c0a3c40 0x7fd08c0a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0103b10 tx=0x7fd09c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:14.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.228+0000 7fd092ffd700 1 -- 172.21.15.26:0/1056011722 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd094013070 con 0x7fd0a0103cb0 2024-04-21T10:28:14.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.435+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/1056011722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fd0a0063650 con 0x7fd0a0103cb0 2024-04-21T10:28:14.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.435+0000 7fd092ffd700 1 -- 172.21.15.26:0/1056011722 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7fd09409de00 con 0x7fd0a0103cb0 2024-04-21T10:28:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:14.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.437+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/1056011722 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd08c0a3c40 msgr2=0x7fd08c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:14.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.437+0000 7fd0a7f0a700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd08c0a3c40 0x7fd08c0a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0103b10 tx=0x7fd09c009190 comp rx=0 tx=0).stop 2024-04-21T10:28:14.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.437+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/1056011722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a0103cb0 msgr2=0x7fd0a0074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:14.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.437+0000 7fd0a7f0a700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a0103cb0 0x7fd0a0074e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd094003790 tx=0x7fd094005b00 comp rx=0 tx=0).stop 2024-04-21T10:28:14.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.437+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/1056011722 shutdown_connections 2024-04-21T10:28:14.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.438+0000 7fd0a7f0a700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd08c0a3c40 0x7fd08c0a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:14.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.438+0000 7fd0a7f0a700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0a0104650 0x7fd0a0075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:14.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.438+0000 7fd0a7f0a700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0a0102ab0 0x7fd0a00767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:14.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.438+0000 7fd0a7f0a700 1 --2- 172.21.15.26:0/1056011722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a0103cb0 0x7fd0a0074e30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:14.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.438+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/1056011722 >> 172.21.15.26:0/1056011722 conn(0x7fd0a00fe010 msgr2=0x7fd0a0106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:14.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.438+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/1056011722 shutdown_connections 2024-04-21T10:28:14.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:14.438+0000 7fd0a7f0a700 1 -- 172.21.15.26:0/1056011722 wait complete. 2024-04-21T10:28:15.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:14 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1056011722' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T10:28:15.116 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:14 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1056011722' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T10:28:15.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:14 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1056011722' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T10:28:15.321 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1713695291,"num_in_osds":12,"osd_in_since":1713695271,"num_remapped_pgs":0} 2024-04-21T10:28:15.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T10:28:15.572 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:28:15.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:15 smithi026 ceph-mon[27965]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:16.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:15 smithi190 ceph-mon[31423]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:16.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:15 smithi169 ceph-mon[31469]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:17.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.185+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1287764950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d406b1f0 msgr2=0x7ff2d406b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:17.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.185+0000 7ff2dbd35700 1 --2- 172.21.15.26:0/1287764950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d406b1f0 0x7ff2d406b670 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009990 tx=0x7ff2d0009ca0 comp rx=0 tx=0).stop 2024-04-21T10:28:17.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.185+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1287764950 shutdown_connections 2024-04-21T10:28:17.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.185+0000 7ff2dbd35700 1 --2- 172.21.15.26:0/1287764950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2d406bbb0 0x7ff2d410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:17.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.185+0000 7ff2dbd35700 1 --2- 172.21.15.26:0/1287764950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d4108a60 0x7ff2d4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:17.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.185+0000 7ff2dbd35700 1 --2- 172.21.15.26:0/1287764950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d406b1f0 0x7ff2d406b670 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:17.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.185+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1287764950 >> 172.21.15.26:0/1287764950 conn(0x7ff2d40789e0 msgr2=0x7ff2d4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:17.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.185+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1287764950 shutdown_connections 2024-04-21T10:28:17.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.185+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1287764950 wait complete. 2024-04-21T10:28:17.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.186+0000 7ff2dbd35700 1 Processor -- start 2024-04-21T10:28:17.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.186+0000 7ff2dbd35700 1 -- start start 2024-04-21T10:28:17.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.186+0000 7ff2dbd35700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2d406b1f0 0x7ff2d41991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:17.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2dbd35700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d406bbb0 0x7ff2d4199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:17.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2dbd35700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d4108a60 0x7ff2d419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:17.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2dbd35700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4112cf0 con 0x7ff2d406bbb0 2024-04-21T10:28:17.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2dbd35700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4112b70 con 0x7ff2d406b1f0 2024-04-21T10:28:17.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2dbd35700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4112e70 con 0x7ff2d4108a60 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2d92d0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d406bbb0 0x7ff2d4199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2d92d0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d406bbb0 0x7ff2d4199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47658/0 (socket says 172.21.15.26:47658) 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2d92d0700 1 -- 172.21.15.26:0/1384187031 learned_addr learned my addr 172.21.15.26:0/1384187031 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2da2d2700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d4108a60 0x7ff2d419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2d9ad1700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2d406b1f0 0x7ff2d41991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2d92d0700 1 -- 172.21.15.26:0/1384187031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d4108a60 msgr2=0x7ff2d419dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2d92d0700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d4108a60 0x7ff2d419dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2d92d0700 1 -- 172.21.15.26:0/1384187031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2d406b1f0 msgr2=0x7ff2d41991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2d92d0700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2d406b1f0 0x7ff2d41991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2d92d0700 1 -- 172.21.15.26:0/1384187031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d419e1f0 con 0x7ff2d406bbb0 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2da2d2700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d4108a60 0x7ff2d419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2d92d0700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d406bbb0 0x7ff2d4199720 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0005910 tx=0x7ff2d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2c6ffd700 1 -- 172.21.15.26:0/1384187031 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0027070 con 0x7ff2d406bbb0 2024-04-21T10:28:17.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1384187031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d419e480 con 0x7ff2d406bbb0 2024-04-21T10:28:17.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2c6ffd700 1 -- 172.21.15.26:0/1384187031 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d0003f20 con 0x7ff2d406bbb0 2024-04-21T10:28:17.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2c6ffd700 1 -- 172.21.15.26:0/1384187031 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0022c50 con 0x7ff2d406bbb0 2024-04-21T10:28:17.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.187+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1384187031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d41a5d60 con 0x7ff2d406bbb0 2024-04-21T10:28:17.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.188+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1384187031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d4069730 con 0x7ff2d406bbb0 2024-04-21T10:28:17.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.190+0000 7ff2c6ffd700 1 -- 172.21.15.26:0/1384187031 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff2d002a070 con 0x7ff2d406bbb0 2024-04-21T10:28:17.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.190+0000 7ff2c6ffd700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff2c00a3c40 0x7ff2c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:17.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.191+0000 7ff2d9ad1700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff2c00a3c40 0x7ff2c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:17.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.191+0000 7ff2c6ffd700 1 -- 172.21.15.26:0/1384187031 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ff2d00d1600 con 0x7ff2d406bbb0 2024-04-21T10:28:17.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.191+0000 7ff2d9ad1700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff2c00a3c40 0x7ff2c00a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8006fd0 tx=0x7ff2c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:17.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.194+0000 7ff2c6ffd700 1 -- 172.21.15.26:0/1384187031 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2d0031070 con 0x7ff2d406bbb0 2024-04-21T10:28:17.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.401+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1384187031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2d419a540 con 0x7ff2d406bbb0 2024-04-21T10:28:17.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.403+0000 7ff2c6ffd700 1 -- 172.21.15.26:0/1384187031 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18444 (secure 0 0 0) 0x7ff2d0013070 con 0x7ff2d406bbb0 2024-04-21T10:28:17.403 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:17.403 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:28:13.521135+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:28:17.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.405+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1384187031 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff2c00a3c40 msgr2=0x7ff2c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:17.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.405+0000 7ff2dbd35700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff2c00a3c40 0x7ff2c00a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8006fd0 tx=0x7ff2c8008040 comp rx=0 tx=0).stop 2024-04-21T10:28:17.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.405+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1384187031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d406bbb0 msgr2=0x7ff2d4199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:17.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.405+0000 7ff2dbd35700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d406bbb0 0x7ff2d4199720 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0005910 tx=0x7ff2d0009f90 comp rx=0 tx=0).stop 2024-04-21T10:28:17.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.405+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1384187031 shutdown_connections 2024-04-21T10:28:17.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.405+0000 7ff2dbd35700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff2c00a3c40 0x7ff2c00a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:17.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.405+0000 7ff2dbd35700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2d406b1f0 0x7ff2d41991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:17.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.405+0000 7ff2dbd35700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2d4108a60 0x7ff2d419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:17.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.405+0000 7ff2dbd35700 1 --2- 172.21.15.26:0/1384187031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2d406bbb0 0x7ff2d4199720 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:17.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.405+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1384187031 >> 172.21.15.26:0/1384187031 conn(0x7ff2d40789e0 msgr2=0x7ff2d40ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:17.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.405+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1384187031 shutdown_connections 2024-04-21T10:28:17.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:17.405+0000 7ff2dbd35700 1 -- 172.21.15.26:0/1384187031 wait complete. 2024-04-21T10:28:18.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:17 smithi190 ceph-mon[31423]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:18.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:17 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1384187031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:28:18.036 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:17 smithi026 ceph-mon[27965]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:18.036 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:17 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1384187031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:28:18.067 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T10:24:28.176891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T10:28:18.067 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-21T10:28:18.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:17 smithi169 ceph-mon[31469]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:18.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:17 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1384187031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:28:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:28:19.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.903+0000 7fa948ba1700 1 -- 172.21.15.26:0/396673118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa944102300 msgr2=0x7fa94410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:19.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.903+0000 7fa948ba1700 1 --2- 172.21.15.26:0/396673118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa944102300 0x7fa94410e7f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00b0a0 tx=0x7fa93c00b3b0 comp rx=0 tx=0).stop 2024-04-21T10:28:19.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.903+0000 7fa948ba1700 1 -- 172.21.15.26:0/396673118 shutdown_connections 2024-04-21T10:28:19.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.903+0000 7fa948ba1700 1 --2- 172.21.15.26:0/396673118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa944107990 0x7fa944107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:19.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.903+0000 7fa948ba1700 1 --2- 172.21.15.26:0/396673118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa944102300 0x7fa94410e7f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:19.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.903+0000 7fa948ba1700 1 --2- 172.21.15.26:0/396673118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa944101940 0x7fa944101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:19.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.903+0000 7fa948ba1700 1 -- 172.21.15.26:0/396673118 >> 172.21.15.26:0/396673118 conn(0x7fa9440788f0 msgr2=0x7fa944078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:19.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.903+0000 7fa948ba1700 1 -- 172.21.15.26:0/396673118 shutdown_connections 2024-04-21T10:28:19.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.903+0000 7fa948ba1700 1 -- 172.21.15.26:0/396673118 wait complete. 2024-04-21T10:28:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.904+0000 7fa948ba1700 1 Processor -- start 2024-04-21T10:28:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.904+0000 7fa948ba1700 1 -- start start 2024-04-21T10:28:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.904+0000 7fa948ba1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa944101940 0x7fa9441990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.904+0000 7fa948ba1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa944102300 0x7fa944199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.904+0000 7fa948ba1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa944107990 0x7fa94419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.904+0000 7fa948ba1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa944111ae0 con 0x7fa944107990 2024-04-21T10:28:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.904+0000 7fa948ba1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa944111960 con 0x7fa944102300 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.904+0000 7fa948ba1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa944111c60 con 0x7fa944101940 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.904+0000 7fa942d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa944107990 0x7fa94419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.904+0000 7fa942d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa944107990 0x7fa94419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47670/0 (socket says 172.21.15.26:47670) 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.904+0000 7fa942d9d700 1 -- 172.21.15.26:0/3385390324 learned_addr learned my addr 172.21.15.26:0/3385390324 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.904+0000 7fa94259c700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa944101940 0x7fa9441990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa941d9b700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa944102300 0x7fa944199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa942d9d700 1 -- 172.21.15.26:0/3385390324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa944101940 msgr2=0x7fa9441990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa942d9d700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa944101940 0x7fa9441990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa942d9d700 1 -- 172.21.15.26:0/3385390324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa944102300 msgr2=0x7fa944199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa942d9d700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa944102300 0x7fa944199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa942d9d700 1 -- 172.21.15.26:0/3385390324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94419e0b0 con 0x7fa944107990 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa941d9b700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa944102300 0x7fa944199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa942d9d700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa944107990 0x7fa94419d970 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00b830 tx=0x7fa93c004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa9337fe700 1 -- 172.21.15.26:0/3385390324 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c021070 con 0x7fa944107990 2024-04-21T10:28:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa948ba1700 1 -- 172.21.15.26:0/3385390324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94419e340 con 0x7fa944107990 2024-04-21T10:28:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa9337fe700 1 -- 172.21.15.26:0/3385390324 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa93c003c00 con 0x7fa944107990 2024-04-21T10:28:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa9337fe700 1 -- 172.21.15.26:0/3385390324 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c004730 con 0x7fa944107990 2024-04-21T10:28:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.905+0000 7fa948ba1700 1 -- 172.21.15.26:0/3385390324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9441a5c20 con 0x7fa944107990 2024-04-21T10:28:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.906+0000 7fa948ba1700 1 -- 172.21.15.26:0/3385390324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa944069730 con 0x7fa944107990 2024-04-21T10:28:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.907+0000 7fa9337fe700 1 -- 172.21.15.26:0/3385390324 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa93c007a00 con 0x7fa944107990 2024-04-21T10:28:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.908+0000 7fa9337fe700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa92c0a3c40 0x7fa92c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.908+0000 7fa94259c700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa92c0a3c40 0x7fa92c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.908+0000 7fa9337fe700 1 -- 172.21.15.26:0/3385390324 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fa93c00f020 con 0x7fa944107990 2024-04-21T10:28:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.909+0000 7fa94259c700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa92c0a3c40 0x7fa92c0a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa934005fd0 tx=0x7fa934005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:19.913+0000 7fa9337fe700 1 -- 172.21.15.26:0/3385390324 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa93c02a020 con 0x7fa944107990 2024-04-21T10:28:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:19 smithi026 ceph-mon[27965]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:20.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:19 smithi190 ceph-mon[31423]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:20.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.118+0000 7fa948ba1700 1 -- 172.21.15.26:0/3385390324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa94419a400 con 0x7fa944107990 2024-04-21T10:28:20.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.118+0000 7fa9337fe700 1 -- 172.21.15.26:0/3385390324 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fa93c09e560 con 0x7fa944107990 2024-04-21T10:28:20.118 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-21T10:28:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.120+0000 7fa948ba1700 1 -- 172.21.15.26:0/3385390324 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa92c0a3c40 msgr2=0x7fa92c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.120+0000 7fa948ba1700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa92c0a3c40 0x7fa92c0a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa934005fd0 tx=0x7fa934005db0 comp rx=0 tx=0).stop 2024-04-21T10:28:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.120+0000 7fa948ba1700 1 -- 172.21.15.26:0/3385390324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa944107990 msgr2=0x7fa94419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.120+0000 7fa948ba1700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa944107990 0x7fa94419d970 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00b830 tx=0x7fa93c004d90 comp rx=0 tx=0).stop 2024-04-21T10:28:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.121+0000 7fa948ba1700 1 -- 172.21.15.26:0/3385390324 shutdown_connections 2024-04-21T10:28:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.121+0000 7fa948ba1700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa92c0a3c40 0x7fa92c0a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.121+0000 7fa948ba1700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa944107990 0x7fa94419d970 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.121+0000 7fa948ba1700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa944101940 0x7fa9441990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.121+0000 7fa948ba1700 1 --2- 172.21.15.26:0/3385390324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa944102300 0x7fa944199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.121+0000 7fa948ba1700 1 -- 172.21.15.26:0/3385390324 >> 172.21.15.26:0/3385390324 conn(0x7fa9440788f0 msgr2=0x7fa94406aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.121+0000 7fa948ba1700 1 -- 172.21.15.26:0/3385390324 shutdown_connections 2024-04-21T10:28:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:20.121+0000 7fa948ba1700 1 -- 172.21.15.26:0/3385390324 wait complete. 2024-04-21T10:28:20.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:19 smithi169 ceph-mon[31469]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:20.851 INFO:tasks.cephadm:Adding mds.a on smithi026 2024-04-21T10:28:20.851 INFO:tasks.cephadm:Adding mds.d on smithi026 2024-04-21T10:28:20.851 INFO:tasks.cephadm:Adding mds.g on smithi026 2024-04-21T10:28:20.851 INFO:tasks.cephadm:Adding mds.j on smithi026 2024-04-21T10:28:20.851 INFO:tasks.cephadm:Adding mds.b on smithi169 2024-04-21T10:28:20.851 INFO:tasks.cephadm:Adding mds.e on smithi169 2024-04-21T10:28:20.851 INFO:tasks.cephadm:Adding mds.h on smithi169 2024-04-21T10:28:20.851 INFO:tasks.cephadm:Adding mds.k on smithi169 2024-04-21T10:28:20.851 INFO:tasks.cephadm:Adding mds.c on smithi190 2024-04-21T10:28:20.851 INFO:tasks.cephadm:Adding mds.f on smithi190 2024-04-21T10:28:20.852 INFO:tasks.cephadm:Adding mds.i on smithi190 2024-04-21T10:28:20.852 INFO:tasks.cephadm:Adding mds.l on smithi190 2024-04-21T10:28:20.852 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l' 2024-04-21T10:28:21.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:20 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3385390324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:28:21.104 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:28:21.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:20 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3385390324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:28:21.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:20 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3385390324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:28:22.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:21 smithi190 ceph-mon[31423]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:22.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:21 smithi026 ceph-mon[27965]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:22.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:21 smithi169 ceph-mon[31469]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:22.645 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.643+0000 7f7529e77700 1 -- 172.21.15.190:0/1506392207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7524103580 msgr2=0x7f7524109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:22.645 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.643+0000 7f7529e77700 1 --2- 172.21.15.190:0/1506392207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7524103580 0x7f7524109e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f751c00b0a0 tx=0x7f751c00b3b0 comp rx=0 tx=0).stop 2024-04-21T10:28:22.645 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.644+0000 7f7529e77700 1 -- 172.21.15.190:0/1506392207 shutdown_connections 2024-04-21T10:28:22.645 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.644+0000 7f7529e77700 1 --2- 172.21.15.190:0/1506392207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7524103580 0x7f7524109e30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:22.645 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.644+0000 7f7529e77700 1 --2- 172.21.15.190:0/1506392207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75241019e0 0x7f7524101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:22.645 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.644+0000 7f7529e77700 1 --2- 172.21.15.190:0/1506392207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7524102be0 0x7f7524103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:22.645 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.644+0000 7f7529e77700 1 -- 172.21.15.190:0/1506392207 >> 172.21.15.190:0/1506392207 conn(0x7f75240fcfa0 msgr2=0x7f75240ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:22.645 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.644+0000 7f7529e77700 1 -- 172.21.15.190:0/1506392207 shutdown_connections 2024-04-21T10:28:22.645 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.644+0000 7f7529e77700 1 -- 172.21.15.190:0/1506392207 wait complete. 2024-04-21T10:28:22.645 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.644+0000 7f7529e77700 1 Processor -- start 2024-04-21T10:28:22.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.644+0000 7f7529e77700 1 -- start start 2024-04-21T10:28:22.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.644+0000 7f7529e77700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75241019e0 0x7f7524198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:22.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.644+0000 7f7529e77700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7524102be0 0x7f7524199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:22.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7529e77700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7524103580 0x7f75241a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:22.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7529e77700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f752410d520 con 0x7f7524103580 2024-04-21T10:28:22.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7529e77700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f752410d3a0 con 0x7f75241019e0 2024-04-21T10:28:22.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7529e77700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f752410d6a0 con 0x7f7524102be0 2024-04-21T10:28:22.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7522ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7524102be0 0x7f7524199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:22.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7522ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7524102be0 0x7f7524199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:54648/0 (socket says 172.21.15.190:54648) 2024-04-21T10:28:22.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7522ffd700 1 -- 172.21.15.190:0/3410724487 learned_addr learned my addr 172.21.15.190:0/3410724487 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-21T10:28:22.646 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f75237fe700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75241019e0 0x7f7524198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:22.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7523fff700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7524103580 0x7f75241a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:22.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7522ffd700 1 -- 172.21.15.190:0/3410724487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75241019e0 msgr2=0x7f7524198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:22.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7522ffd700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75241019e0 0x7f7524198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:22.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7522ffd700 1 -- 172.21.15.190:0/3410724487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7524103580 msgr2=0x7f75241a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:22.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7522ffd700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7524103580 0x7f75241a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:22.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7522ffd700 1 -- 172.21.15.190:0/3410724487 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75241a08f0 con 0x7f7524102be0 2024-04-21T10:28:22.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7522ffd700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7524102be0 0x7f7524199200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f751400b530 tx=0x7f751400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:22.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7520ff9700 1 -- 172.21.15.190:0/3410724487 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514015020 con 0x7f7524102be0 2024-04-21T10:28:22.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7529e77700 1 -- 172.21.15.190:0/3410724487 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75241a0be0 con 0x7f7524102be0 2024-04-21T10:28:22.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7520ff9700 1 -- 172.21.15.190:0/3410724487 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7514011ce0 con 0x7f7524102be0 2024-04-21T10:28:22.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7520ff9700 1 -- 172.21.15.190:0/3410724487 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514010dd0 con 0x7f7524102be0 2024-04-21T10:28:22.647 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.645+0000 7f7529e77700 1 -- 172.21.15.190:0/3410724487 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75241a1120 con 0x7f7524102be0 2024-04-21T10:28:22.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.646+0000 7f7529e77700 1 -- 172.21.15.190:0/3410724487 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7524069760 con 0x7f7524102be0 2024-04-21T10:28:22.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.647+0000 7f7520ff9700 1 -- 172.21.15.190:0/3410724487 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7514018020 con 0x7f7524102be0 2024-04-21T10:28:22.648 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.648+0000 7f7520ff9700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f750c0a3c40 0x7f750c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:22.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.648+0000 7f7520ff9700 1 -- 172.21.15.190:0/3410724487 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f75140cf630 con 0x7f7524102be0 2024-04-21T10:28:22.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.648+0000 7f75237fe700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f750c0a3c40 0x7f750c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:22.649 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.649+0000 7f75237fe700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f750c0a3c40 0x7f750c0a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f752410a340 tx=0x7f751800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:22.652 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.653+0000 7f7520ff9700 1 -- 172.21.15.190:0/3410724487 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7524069760 con 0x7f7524102be0 2024-04-21T10:28:22.859 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.859+0000 7f7529e77700 1 -- 172.21.15.190:0/3410724487 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}) -- 0x7f75240ff270 con 0x7f750c0a3c40 2024-04-21T10:28:22.867 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.867+0000 7f7520ff9700 1 -- 172.21.15.190:0/3410724487 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f75240ff270 con 0x7f750c0a3c40 2024-04-21T10:28:22.867 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mds.all update... 2024-04-21T10:28:22.869 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.870+0000 7f7529e77700 1 -- 172.21.15.190:0/3410724487 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f750c0a3c40 msgr2=0x7f750c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:22.870 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.870+0000 7f7529e77700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f750c0a3c40 0x7f750c0a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f752410a340 tx=0x7f751800b410 comp rx=0 tx=0).stop 2024-04-21T10:28:22.870 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.870+0000 7f7529e77700 1 -- 172.21.15.190:0/3410724487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7524102be0 msgr2=0x7f7524199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:22.870 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.870+0000 7f7529e77700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7524102be0 0x7f7524199200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f751400b530 tx=0x7f751400b840 comp rx=0 tx=0).stop 2024-04-21T10:28:22.870 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.870+0000 7f7529e77700 1 -- 172.21.15.190:0/3410724487 shutdown_connections 2024-04-21T10:28:22.870 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.870+0000 7f7529e77700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f750c0a3c40 0x7f750c0a6100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:22.870 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.870+0000 7f7529e77700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75241019e0 0x7f7524198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:22.870 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.870+0000 7f7529e77700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7524103580 0x7f75241a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:22.870 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.870+0000 7f7529e77700 1 --2- 172.21.15.190:0/3410724487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7524102be0 0x7f7524199200 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:22.870 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.870+0000 7f7529e77700 1 -- 172.21.15.190:0/3410724487 >> 172.21.15.190:0/3410724487 conn(0x7f75240fcfa0 msgr2=0x7f75240fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:22.870 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.870+0000 7f7529e77700 1 -- 172.21.15.190:0/3410724487 shutdown_connections 2024-04-21T10:28:22.870 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-21T10:28:22.870+0000 7f7529e77700 1 -- 172.21.15.190:0/3410724487 wait complete. 2024-04-21T10:28:23.472 DEBUG:teuthology.orchestra.run.smithi026:mds.a> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.a.service 2024-04-21T10:28:23.475 DEBUG:teuthology.orchestra.run.smithi026:mds.d> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.d.service 2024-04-21T10:28:23.517 DEBUG:teuthology.orchestra.run.smithi026:mds.g> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.g.service 2024-04-21T10:28:23.562 DEBUG:teuthology.orchestra.run.smithi026:mds.j> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.j.service 2024-04-21T10:28:23.604 DEBUG:teuthology.orchestra.run.smithi169:mds.b> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.b.service 2024-04-21T10:28:23.607 DEBUG:teuthology.orchestra.run.smithi169:mds.e> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.e.service 2024-04-21T10:28:23.649 DEBUG:teuthology.orchestra.run.smithi169:mds.h> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.h.service 2024-04-21T10:28:23.692 DEBUG:teuthology.orchestra.run.smithi169:mds.k> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.k.service 2024-04-21T10:28:23.735 DEBUG:teuthology.orchestra.run.smithi190:mds.c> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.c.service 2024-04-21T10:28:23.737 DEBUG:teuthology.orchestra.run.smithi190:mds.f> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.f.service 2024-04-21T10:28:23.780 DEBUG:teuthology.orchestra.run.smithi190:mds.i> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.i.service 2024-04-21T10:28:23.823 DEBUG:teuthology.orchestra.run.smithi190:mds.l> sudo journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.l.service 2024-04-21T10:28:23.867 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-21T10:28:23.867 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:28:23.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:23 smithi169 ceph-mon[31469]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:23.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:23 smithi169 ceph-mon[31469]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:28:23.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:23 smithi169 ceph-mon[31469]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-04-21T10:28:23.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:23.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:23.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:23.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:28:23.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:23.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:23.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:23.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:23 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.982+0000 7fbb43a26640 1 -- 172.21.15.26:0/454706189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb3c103b30 msgr2=0x7fbb3c10a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.982+0000 7fbb43a26640 1 --2- 172.21.15.26:0/454706189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb3c103b30 0x7fbb3c10a3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009be0 tx=0x7fbb2c02bef0 comp rx=0 tx=0).stop 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.982+0000 7fbb43a26640 1 -- 172.21.15.26:0/454706189 shutdown_connections 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.982+0000 7fbb43a26640 1 --2- 172.21.15.26:0/454706189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb3c103b30 0x7fbb3c10a3c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.982+0000 7fbb43a26640 1 --2- 172.21.15.26:0/454706189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c103170 0x7fbb3c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.982+0000 7fbb43a26640 1 --2- 172.21.15.26:0/454706189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb3c101f80 0x7fbb3c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.982+0000 7fbb43a26640 1 -- 172.21.15.26:0/454706189 >> 172.21.15.26:0/454706189 conn(0x7fbb3c0fd530 msgr2=0x7fbb3c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.982+0000 7fbb43a26640 1 -- 172.21.15.26:0/454706189 shutdown_connections 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.982+0000 7fbb43a26640 1 -- 172.21.15.26:0/454706189 wait complete. 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb43a26640 1 Processor -- start 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb43a26640 1 -- start start 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb43a26640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb3c101f80 0x7fbb3c075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb43a26640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb3c103170 0x7fbb3c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb43a26640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c103b30 0x7fbb3c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb43a26640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c10db10 con 0x7fbb3c103170 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb43a26640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c10d990 con 0x7fbb3c103b30 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb43a26640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c10dc90 con 0x7fbb3c101f80 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb40fc1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb3c103170 0x7fbb3c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb40fc1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb3c103170 0x7fbb3c074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47686/0 (socket says 172.21.15.26:47686) 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb40fc1640 1 -- 172.21.15.26:0/1761094046 learned_addr learned my addr 172.21.15.26:0/1761094046 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb417c2640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb3c101f80 0x7fbb3c075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb41fc3640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c103b30 0x7fbb3c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb40fc1640 1 -- 172.21.15.26:0/1761094046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb3c101f80 msgr2=0x7fbb3c075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb40fc1640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb3c101f80 0x7fbb3c075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb40fc1640 1 -- 172.21.15.26:0/1761094046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c103b30 msgr2=0x7fbb3c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb40fc1640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c103b30 0x7fbb3c074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb40fc1640 1 -- 172.21.15.26:0/1761094046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c075110 con 0x7fbb3c103170 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.983+0000 7fbb40fc1640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb3c103170 0x7fbb3c074340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000e9f0 tx=0x7fbb3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.984+0000 7fbb3a7fc640 1 -- 172.21.15.26:0/1761094046 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3000ccc0 con 0x7fbb3c103170 2024-04-21T10:28:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.984+0000 7fbb43a26640 1 -- 172.21.15.26:0/1761094046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3c1a22b0 con 0x7fbb3c103170 2024-04-21T10:28:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.984+0000 7fbb3a7fc640 1 -- 172.21.15.26:0/1761094046 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb300108e0 con 0x7fbb3c103170 2024-04-21T10:28:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.984+0000 7fbb3a7fc640 1 -- 172.21.15.26:0/1761094046 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30018880 con 0x7fbb3c103170 2024-04-21T10:28:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.984+0000 7fbb43a26640 1 -- 172.21.15.26:0/1761094046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3c1a2800 con 0x7fbb3c103170 2024-04-21T10:28:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.985+0000 7fbb43a26640 1 -- 172.21.15.26:0/1761094046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3c1092f0 con 0x7fbb3c103170 2024-04-21T10:28:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.986+0000 7fbb3a7fc640 1 -- 172.21.15.26:0/1761094046 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbb30010430 con 0x7fbb3c103170 2024-04-21T10:28:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.986+0000 7fbb3a7fc640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbb240a3bb0 0x7fbb240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.987+0000 7fbb417c2640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbb240a3bb0 0x7fbb240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.987+0000 7fbb3a7fc640 1 -- 172.21.15.26:0/1761094046 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fbb300d03b0 con 0x7fbb3c103170 2024-04-21T10:28:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.987+0000 7fbb417c2640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbb240a3bb0 0x7fbb240a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c102fd0 tx=0x7fbb34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:23.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:23.991+0000 7fbb3a7fc640 1 -- 172.21.15.26:0/1761094046 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb3009d230 con 0x7fbb3c103170 2024-04-21T10:28:24.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:23 smithi190 ceph-mon[31423]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:23 smithi190 ceph-mon[31423]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:28:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:23 smithi190 ceph-mon[31423]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-04-21T10:28:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:28:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:24.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:23 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:24.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.179+0000 7fbb43a26640 1 -- 172.21.15.26:0/1761094046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb3c02d830 con 0x7fbb3c103170 2024-04-21T10:28:24.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.180+0000 7fbb3a7fc640 1 -- 172.21.15.26:0/1761094046 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18444 (secure 0 0 0) 0x7fbb3009cb30 con 0x7fbb3c103170 2024-04-21T10:28:24.180 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:24.180 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:28:13.521135+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:28:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.182+0000 7fbb43a26640 1 -- 172.21.15.26:0/1761094046 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbb240a3bb0 msgr2=0x7fbb240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.182+0000 7fbb43a26640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbb240a3bb0 0x7fbb240a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c102fd0 tx=0x7fbb34009260 comp rx=0 tx=0).stop 2024-04-21T10:28:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.182+0000 7fbb43a26640 1 -- 172.21.15.26:0/1761094046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb3c103170 msgr2=0x7fbb3c074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.182+0000 7fbb43a26640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb3c103170 0x7fbb3c074340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000e9f0 tx=0x7fbb3000edf0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.182+0000 7fbb43a26640 1 -- 172.21.15.26:0/1761094046 shutdown_connections 2024-04-21T10:28:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.182+0000 7fbb43a26640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbb240a3bb0 0x7fbb240a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.182+0000 7fbb43a26640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3c103b30 0x7fbb3c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.182+0000 7fbb43a26640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb3c103170 0x7fbb3c074340 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.182+0000 7fbb43a26640 1 --2- 172.21.15.26:0/1761094046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb3c101f80 0x7fbb3c075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.182+0000 7fbb43a26640 1 -- 172.21.15.26:0/1761094046 >> 172.21.15.26:0/1761094046 conn(0x7fbb3c0fd530 msgr2=0x7fbb3c1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.182+0000 7fbb43a26640 1 -- 172.21.15.26:0/1761094046 shutdown_connections 2024-04-21T10:28:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.182+0000 7fbb43a26640 1 -- 172.21.15.26:0/1761094046 wait complete. 2024-04-21T10:28:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:23 smithi026 ceph-mon[27965]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:23 smithi026 ceph-mon[27965]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:28:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:23 smithi026 ceph-mon[27965]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-04-21T10:28:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:28:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:24.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:23 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:24.193 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T10:24:28.176891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T10:28:24.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T10:28:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.299+0000 7f625ae8c640 1 -- 172.21.15.26:0/2637382693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6254102a30 msgr2=0x7f6254104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.299+0000 7f625ae8c640 1 --2- 172.21.15.26:0/2637382693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6254102a30 0x7f6254104e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6244009900 tx=0x7f624402bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.299+0000 7f625ae8c640 1 -- 172.21.15.26:0/2637382693 shutdown_connections 2024-04-21T10:28:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.299+0000 7f625ae8c640 1 --2- 172.21.15.26:0/2637382693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6254107c90 0x7f625410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.299+0000 7f625ae8c640 1 --2- 172.21.15.26:0/2637382693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6254105360 0x7f6254107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.299+0000 7f625ae8c640 1 --2- 172.21.15.26:0/2637382693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6254102a30 0x7f6254104e20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.299+0000 7f625ae8c640 1 -- 172.21.15.26:0/2637382693 >> 172.21.15.26:0/2637382693 conn(0x7f62540fc440 msgr2=0x7f62540fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.299+0000 7f625ae8c640 1 -- 172.21.15.26:0/2637382693 shutdown_connections 2024-04-21T10:28:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.299+0000 7f625ae8c640 1 -- 172.21.15.26:0/2637382693 wait complete. 2024-04-21T10:28:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.299+0000 7f625ae8c640 1 Processor -- start 2024-04-21T10:28:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.299+0000 7f625ae8c640 1 -- start start 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f625ae8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6254102a30 0x7f6254198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f625ae8c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6254105360 0x7f62541987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f625ae8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6254107c90 0x7f625419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f625ae8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f625410db50 con 0x7f6254107c90 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f625ae8c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f625410d9d0 con 0x7f6254105360 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f625ae8c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f625410dcd0 con 0x7f6254102a30 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6259429640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6254107c90 0x7f625419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6258c28640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6254102a30 0x7f6254198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6259429640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6254107c90 0x7f625419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47698/0 (socket says 172.21.15.26:47698) 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6259429640 1 -- 172.21.15.26:0/347440464 learned_addr learned my addr 172.21.15.26:0/347440464 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6253fff640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6254105360 0x7f62541987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6259429640 1 -- 172.21.15.26:0/347440464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6254102a30 msgr2=0x7f6254198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6259429640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6254102a30 0x7f6254198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6259429640 1 -- 172.21.15.26:0/347440464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6254105360 msgr2=0x7f62541987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6259429640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6254105360 0x7f62541987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6259429640 1 -- 172.21.15.26:0/347440464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625419fea0 con 0x7f6254107c90 2024-04-21T10:28:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6253fff640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6254105360 0x7f62541987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:28:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6259429640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6254107c90 0x7f625419f830 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f624c00b970 tx=0x7f624c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6251ffb640 1 -- 172.21.15.26:0/347440464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c019070 con 0x7f6254107c90 2024-04-21T10:28:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6251ffb640 1 -- 172.21.15.26:0/347440464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f624c014440 con 0x7f6254107c90 2024-04-21T10:28:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f625ae8c640 1 -- 172.21.15.26:0/347440464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62541a0190 con 0x7f6254107c90 2024-04-21T10:28:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.300+0000 7f6251ffb640 1 -- 172.21.15.26:0/347440464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c0075d0 con 0x7f6254107c90 2024-04-21T10:28:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.301+0000 7f625ae8c640 1 -- 172.21.15.26:0/347440464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62541a0760 con 0x7f6254107c90 2024-04-21T10:28:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.302+0000 7f625ae8c640 1 -- 172.21.15.26:0/347440464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62540fdf30 con 0x7f6254107c90 2024-04-21T10:28:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.303+0000 7f6251ffb640 1 -- 172.21.15.26:0/347440464 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f624c01c070 con 0x7f6254107c90 2024-04-21T10:28:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.303+0000 7f6251ffb640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f62400a3c30 0x7f62400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.303+0000 7f6258c28640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f62400a3c30 0x7f62400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.303+0000 7f6251ffb640 1 -- 172.21.15.26:0/347440464 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f624c0cf520 con 0x7f6254107c90 2024-04-21T10:28:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.304+0000 7f6258c28640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f62400a3c30 0x7f62400a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6244009230 tx=0x7f6244005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.308+0000 7f6251ffb640 1 -- 172.21.15.26:0/347440464 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f624c09bcb0 con 0x7f6254107c90 2024-04-21T10:28:24.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.492+0000 7f625ae8c640 1 -- 172.21.15.26:0/347440464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6254062b50 con 0x7f6254107c90 2024-04-21T10:28:24.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.492+0000 7f6251ffb640 1 -- 172.21.15.26:0/347440464 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f624c09ffb0 con 0x7f6254107c90 2024-04-21T10:28:24.492 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-21T10:28:24.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.494+0000 7f625ae8c640 1 -- 172.21.15.26:0/347440464 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f62400a3c30 msgr2=0x7f62400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:24.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.494+0000 7f625ae8c640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f62400a3c30 0x7f62400a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6244009230 tx=0x7f6244005fd0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.494+0000 7f625ae8c640 1 -- 172.21.15.26:0/347440464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6254107c90 msgr2=0x7f625419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:24.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.494+0000 7f625ae8c640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6254107c90 0x7f625419f830 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f624c00b970 tx=0x7f624c00be30 comp rx=0 tx=0).stop 2024-04-21T10:28:24.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.494+0000 7f625ae8c640 1 -- 172.21.15.26:0/347440464 shutdown_connections 2024-04-21T10:28:24.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.494+0000 7f625ae8c640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f62400a3c30 0x7f62400a60f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.494+0000 7f625ae8c640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6254107c90 0x7f625419f830 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.494+0000 7f625ae8c640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6254105360 0x7f62541987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.494+0000 7f625ae8c640 1 --2- 172.21.15.26:0/347440464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6254102a30 0x7f6254198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.495+0000 7f625ae8c640 1 -- 172.21.15.26:0/347440464 >> 172.21.15.26:0/347440464 conn(0x7f62540fc440 msgr2=0x7f6254103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:24.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.495+0000 7f625ae8c640 1 -- 172.21.15.26:0/347440464 shutdown_connections 2024-04-21T10:28:24.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.495+0000 7f625ae8c640 1 -- 172.21.15.26:0/347440464 wait complete. 2024-04-21T10:28:24.506 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:24.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.607+0000 7fcb18621640 1 -- 172.21.15.26:0/393636966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb10107ca0 msgr2=0x7fcb1010a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:24.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.607+0000 7fcb18621640 1 --2- 172.21.15.26:0/393636966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb10107ca0 0x7fcb1010a0f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009900 tx=0x7fcb0802bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.607+0000 7fcb18621640 1 -- 172.21.15.26:0/393636966 shutdown_connections 2024-04-21T10:28:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.607+0000 7fcb18621640 1 --2- 172.21.15.26:0/393636966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb10107ca0 0x7fcb1010a0f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.607+0000 7fcb18621640 1 --2- 172.21.15.26:0/393636966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb10105370 0x7fcb10107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.607+0000 7fcb18621640 1 --2- 172.21.15.26:0/393636966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb10102a40 0x7fcb10104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.607+0000 7fcb18621640 1 -- 172.21.15.26:0/393636966 >> 172.21.15.26:0/393636966 conn(0x7fcb100fc470 msgr2=0x7fcb100fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb18621640 1 -- 172.21.15.26:0/393636966 shutdown_connections 2024-04-21T10:28:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb18621640 1 -- 172.21.15.26:0/393636966 wait complete. 2024-04-21T10:28:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb18621640 1 Processor -- start 2024-04-21T10:28:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb18621640 1 -- start start 2024-04-21T10:28:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb18621640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb10102a40 0x7fcb101980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb18621640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb10105370 0x7fcb101985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb18621640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb10107ca0 0x7fcb1019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb18621640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010db00 con 0x7fcb10105370 2024-04-21T10:28:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb18621640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010d980 con 0x7fcb10102a40 2024-04-21T10:28:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb18621640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010dc80 con 0x7fcb10107ca0 2024-04-21T10:28:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb15bbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb10105370 0x7fcb101985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb15bbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb10105370 0x7fcb101985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44300/0 (socket says 172.21.15.26:44300) 2024-04-21T10:28:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb15bbc640 1 -- 172.21.15.26:0/3649511347 learned_addr learned my addr 172.21.15.26:0/3649511347 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb163bd640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb10102a40 0x7fcb101980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.608+0000 7fcb16bbe640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb10107ca0 0x7fcb1019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.609+0000 7fcb15bbc640 1 -- 172.21.15.26:0/3649511347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb10107ca0 msgr2=0x7fcb1019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.609+0000 7fcb15bbc640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb10107ca0 0x7fcb1019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.609+0000 7fcb15bbc640 1 -- 172.21.15.26:0/3649511347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb10102a40 msgr2=0x7fcb101980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.609+0000 7fcb15bbc640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb10102a40 0x7fcb101980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.609+0000 7fcb15bbc640 1 -- 172.21.15.26:0/3649511347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1019fcd0 con 0x7fcb10105370 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.609+0000 7fcb15bbc640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb10105370 0x7fcb101985f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000c970 tx=0x7fcb0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.609+0000 7fcb077fe640 1 -- 172.21.15.26:0/3649511347 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00014070 con 0x7fcb10105370 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.609+0000 7fcb18621640 1 -- 172.21.15.26:0/3649511347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1019ffc0 con 0x7fcb10105370 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.609+0000 7fcb077fe640 1 -- 172.21.15.26:0/3649511347 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb00004440 con 0x7fcb10105370 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.609+0000 7fcb077fe640 1 -- 172.21.15.26:0/3649511347 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00003e50 con 0x7fcb10105370 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.609+0000 7fcb18621640 1 -- 172.21.15.26:0/3649511347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101a0480 con 0x7fcb10105370 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.610+0000 7fcb18621640 1 -- 172.21.15.26:0/3649511347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb100fdf60 con 0x7fcb10105370 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.611+0000 7fcb077fe640 1 -- 172.21.15.26:0/3649511347 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcb00017070 con 0x7fcb10105370 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.612+0000 7fcb077fe640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcafc0a3d50 0x7fcafc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.612+0000 7fcb163bd640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcafc0a3d50 0x7fcafc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:24.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.612+0000 7fcb077fe640 1 -- 172.21.15.26:0/3649511347 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fcb000cfb40 con 0x7fcb10105370 2024-04-21T10:28:24.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.612+0000 7fcb163bd640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcafc0a3d50 0x7fcafc0a6210 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c005fd0 tx=0x7fcb0c009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:24.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.616+0000 7fcb077fe640 1 -- 172.21.15.26:0/3649511347 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb0001e030 con 0x7fcb10105370 2024-04-21T10:28:24.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.844+0000 7fcb18621640 1 -- 172.21.15.26:0/3649511347 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb1002d5c0 con 0x7fcb10105370 2024-04-21T10:28:24.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.844+0000 7fcb077fe640 1 -- 172.21.15.26:0/3649511347 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fcb0000de20 con 0x7fcb10105370 2024-04-21T10:28:24.844 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:24.844 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-21T10:28:24.844 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 1 2024-04-21T10:28:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.846+0000 7fcb18621640 1 -- 172.21.15.26:0/3649511347 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcafc0a3d50 msgr2=0x7fcafc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.846+0000 7fcb18621640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcafc0a3d50 0x7fcafc0a6210 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c005fd0 tx=0x7fcb0c009520 comp rx=0 tx=0).stop 2024-04-21T10:28:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.846+0000 7fcb18621640 1 -- 172.21.15.26:0/3649511347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb10105370 msgr2=0x7fcb101985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.846+0000 7fcb18621640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb10105370 0x7fcb101985f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000c970 tx=0x7fcb0000ce30 comp rx=0 tx=0).stop 2024-04-21T10:28:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.846+0000 7fcb18621640 1 -- 172.21.15.26:0/3649511347 shutdown_connections 2024-04-21T10:28:24.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.846+0000 7fcb18621640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcafc0a3d50 0x7fcafc0a6210 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.846+0000 7fcb18621640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb10107ca0 0x7fcb1019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.846+0000 7fcb18621640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb10105370 0x7fcb101985f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.846+0000 7fcb18621640 1 --2- 172.21.15.26:0/3649511347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb10102a40 0x7fcb101980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:24.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.846+0000 7fcb18621640 1 -- 172.21.15.26:0/3649511347 >> 172.21.15.26:0/3649511347 conn(0x7fcb100fc470 msgr2=0x7fcb10103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:24.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.846+0000 7fcb18621640 1 -- 172.21.15.26:0/3649511347 shutdown_connections 2024-04-21T10:28:24.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:24.846+0000 7fcb18621640 1 -- 172.21.15.26:0/3649511347 wait complete. 2024-04-21T10:28:25.038 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:24 smithi190 ceph-mon[31423]: Deploying daemon mds.l on smithi190 2024-04-21T10:28:25.038 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:24 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1761094046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:28:25.038 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:24 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/347440464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:28:25.039 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:24 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3649511347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:25.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:24 smithi026 ceph-mon[27965]: Deploying daemon mds.l on smithi190 2024-04-21T10:28:25.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:24 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1761094046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:28:25.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:24 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/347440464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:28:25.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:24 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3649511347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:25.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:24 smithi169 ceph-mon[31469]: Deploying daemon mds.l on smithi190 2024-04-21T10:28:25.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:24 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1761094046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:28:25.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:24 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/347440464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:28:25.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:24 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3649511347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:25.709 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:28:25 smithi190 systemd[1]: Starting Ceph mds.l for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:28:25.959 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:25 smithi190 ceph-mon[31423]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:25.960 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:28:25 smithi190 podman[61797]: 2024-04-21 10:28:25.612625463 +0000 UTC m=+0.017988170 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:28:25.960 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:28:25 smithi190 podman[61797]: 2024-04-21 10:28:25.733759966 +0000 UTC m=+0.139122648 container create 68236a017c6d51094348f5660c6f5dec09140f8b839d845ef07c0b889965ce52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-21T10:28:25.960 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:28:25 smithi190 podman[61797]: 2024-04-21 10:28:25.96112527 +0000 UTC m=+0.366487953 container init 68236a017c6d51094348f5660c6f5dec09140f8b839d845ef07c0b889965ce52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:28:26.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:25 smithi026 ceph-mon[27965]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:26.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:25 smithi169 ceph-mon[31469]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:26.276 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:28:25 smithi190 podman[61797]: 2024-04-21 10:28:25.963917601 +0000 UTC m=+0.369280287 container start 68236a017c6d51094348f5660c6f5dec09140f8b839d845ef07c0b889965ce52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-21T10:28:26.277 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:28:25 smithi190 ceph-mds[61816]: -- 172.21.15.190:0/3908845911 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b62c45a40 con 0x563b62c19000 2024-04-21T10:28:26.277 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:28:25 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-l[61812]: starting mds.l at 2024-04-21T10:28:26.277 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:28:26 smithi190 bash[61797]: 68236a017c6d51094348f5660c6f5dec09140f8b839d845ef07c0b889965ce52 2024-04-21T10:28:26.277 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:28:26 smithi190 systemd[1]: Started Ceph mds.l for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:28:26.858 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.973+0000 7fcdefaba640 1 -- 172.21.15.26:0/398180854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8103b50 msgr2=0x7fcde810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.973+0000 7fcdefaba640 1 --2- 172.21.15.26:0/398180854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8103b50 0x7fcde810a3e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009ca0 tx=0x7fcde002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:28:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.973+0000 7fcdefaba640 1 -- 172.21.15.26:0/398180854 shutdown_connections 2024-04-21T10:28:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.973+0000 7fcdefaba640 1 --2- 172.21.15.26:0/398180854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8103b50 0x7fcde810a3e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.973+0000 7fcdefaba640 1 --2- 172.21.15.26:0/398180854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcde8103190 0x7fcde8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.973+0000 7fcdefaba640 1 --2- 172.21.15.26:0/398180854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde8101fa0 0x7fcde81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.973+0000 7fcdefaba640 1 -- 172.21.15.26:0/398180854 >> 172.21.15.26:0/398180854 conn(0x7fcde80fd530 msgr2=0x7fcde80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.973+0000 7fcdefaba640 1 -- 172.21.15.26:0/398180854 shutdown_connections 2024-04-21T10:28:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.973+0000 7fcdefaba640 1 -- 172.21.15.26:0/398180854 wait complete. 2024-04-21T10:28:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.973+0000 7fcdefaba640 1 Processor -- start 2024-04-21T10:28:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcdefaba640 1 -- start start 2024-04-21T10:28:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcdefaba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8101fa0 0x7fcde8075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcdefaba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcde8103190 0x7fcde8074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcdefaba640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde8103b50 0x7fcde8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcdefaba640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcde810dae0 con 0x7fcde8101fa0 2024-04-21T10:28:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcdefaba640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcde810d960 con 0x7fcde8103b50 2024-04-21T10:28:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcdefaba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcde810dc60 con 0x7fcde8103190 2024-04-21T10:28:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcded856640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8101fa0 0x7fcde8075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcded856640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8101fa0 0x7fcde8075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44324/0 (socket says 172.21.15.26:44324) 2024-04-21T10:28:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcded856640 1 -- 172.21.15.26:0/4128559367 learned_addr learned my addr 172.21.15.26:0/4128559367 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcdee057640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde8103b50 0x7fcde8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcded055640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcde8103190 0x7fcde8074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcded856640 1 -- 172.21.15.26:0/4128559367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcde8103190 msgr2=0x7fcde8074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcded856640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcde8103190 0x7fcde8074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcded856640 1 -- 172.21.15.26:0/4128559367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde8103b50 msgr2=0x7fcde8074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcded856640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde8103b50 0x7fcde8074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcded856640 1 -- 172.21.15.26:0/4128559367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde8075110 con 0x7fcde8101fa0 2024-04-21T10:28:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcded055640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcde8103190 0x7fcde8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:28:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcded856640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8101fa0 0x7fcde8075d00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcde400e9c0 tx=0x7fcde400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.974+0000 7fcddeffd640 1 -- 172.21.15.26:0/4128559367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde400cd80 con 0x7fcde8101fa0 2024-04-21T10:28:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.975+0000 7fcddeffd640 1 -- 172.21.15.26:0/4128559367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde40035f0 con 0x7fcde8101fa0 2024-04-21T10:28:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.975+0000 7fcdefaba640 1 -- 172.21.15.26:0/4128559367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde81a2250 con 0x7fcde8101fa0 2024-04-21T10:28:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.975+0000 7fcddeffd640 1 -- 172.21.15.26:0/4128559367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4010690 con 0x7fcde8101fa0 2024-04-21T10:28:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.975+0000 7fcdefaba640 1 -- 172.21.15.26:0/4128559367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde81a27f0 con 0x7fcde8101fa0 2024-04-21T10:28:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.976+0000 7fcdefaba640 1 -- 172.21.15.26:0/4128559367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde81092c0 con 0x7fcde8101fa0 2024-04-21T10:28:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.977+0000 7fcddeffd640 1 -- 172.21.15.26:0/4128559367 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcde40037a0 con 0x7fcde8101fa0 2024-04-21T10:28:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.977+0000 7fcddeffd640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcdd40a3c30 0x7fcdd40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.977+0000 7fcded055640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcdd40a3c30 0x7fcdd40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.978+0000 7fcddeffd640 1 -- 172.21.15.26:0/4128559367 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fcde40d0350 con 0x7fcde8101fa0 2024-04-21T10:28:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.978+0000 7fcded055640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcdd40a3c30 0x7fcdd40a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcde8075a70 tx=0x7fcdd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:26.982+0000 7fcddeffd640 1 -- 172.21.15.26:0/4128559367 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcde409d1d0 con 0x7fcde8101fa0 2024-04-21T10:28:27.087 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:26 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.190:6832/2081925034,v1:172.21.15.190:6833/2081925034] up:boot 2024-04-21T10:28:27.087 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:26 smithi169 ceph-mon[31469]: fsmap 1 up:standby 2024-04-21T10:28:27.087 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:26 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T10:28:27.087 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:26 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:27.087 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:26 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:27.087 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:26 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:27.087 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:26 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:27.088 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:26 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:27.088 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:26 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:27.088 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:26 smithi169 ceph-mon[31469]: Deploying daemon mds.k on smithi169 2024-04-21T10:28:27.088 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:26 smithi169 ceph-mon[31469]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:27.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:26 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.190:6832/2081925034,v1:172.21.15.190:6833/2081925034] up:boot 2024-04-21T10:28:27.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:26 smithi026 ceph-mon[27965]: fsmap 1 up:standby 2024-04-21T10:28:27.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:26 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T10:28:27.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:26 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:27.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:26 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:27.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:26 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:27.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:26 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:27.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:26 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:27.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:26 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:27.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:26 smithi026 ceph-mon[27965]: Deploying daemon mds.k on smithi169 2024-04-21T10:28:27.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:26 smithi026 ceph-mon[27965]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.209+0000 7fcdefaba640 1 -- 172.21.15.26:0/4128559367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcde802d5c0 con 0x7fcde8101fa0 2024-04-21T10:28:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.210+0000 7fcddeffd640 1 -- 172.21.15.26:0/4128559367 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fcde409cad0 con 0x7fcde8101fa0 2024-04-21T10:28:27.209 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:27.210 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T10:28:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 2 2024-04-21T10:28:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.211+0000 7fcdefaba640 1 -- 172.21.15.26:0/4128559367 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcdd40a3c30 msgr2=0x7fcdd40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.211+0000 7fcdefaba640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcdd40a3c30 0x7fcdd40a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcde8075a70 tx=0x7fcdd8005f90 comp rx=0 tx=0).stop 2024-04-21T10:28:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.212+0000 7fcdefaba640 1 -- 172.21.15.26:0/4128559367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8101fa0 msgr2=0x7fcde8075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.212+0000 7fcdefaba640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8101fa0 0x7fcde8075d00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcde400e9c0 tx=0x7fcde400ee80 comp rx=0 tx=0).stop 2024-04-21T10:28:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.212+0000 7fcdefaba640 1 -- 172.21.15.26:0/4128559367 shutdown_connections 2024-04-21T10:28:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.212+0000 7fcdefaba640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcdd40a3c30 0x7fcdd40a60f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.212+0000 7fcdefaba640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde8103b50 0x7fcde8074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.212+0000 7fcdefaba640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcde8103190 0x7fcde8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.212+0000 7fcdefaba640 1 --2- 172.21.15.26:0/4128559367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8101fa0 0x7fcde8075d00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.212+0000 7fcdefaba640 1 -- 172.21.15.26:0/4128559367 >> 172.21.15.26:0/4128559367 conn(0x7fcde80fd530 msgr2=0x7fcde8106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.212+0000 7fcdefaba640 1 -- 172.21.15.26:0/4128559367 shutdown_connections 2024-04-21T10:28:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:27.212+0000 7fcdefaba640 1 -- 172.21.15.26:0/4128559367 wait complete. 2024-04-21T10:28:27.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:26 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.190:6832/2081925034,v1:172.21.15.190:6833/2081925034] up:boot 2024-04-21T10:28:27.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:26 smithi190 ceph-mon[31423]: fsmap 1 up:standby 2024-04-21T10:28:27.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:26 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T10:28:27.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:26 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:27.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:26 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:27.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:26 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:27.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:26 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:27.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:26 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:27.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:26 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:27.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:26 smithi190 ceph-mon[31423]: Deploying daemon mds.k on smithi169 2024-04-21T10:28:27.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:26 smithi190 ceph-mon[31423]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:28.164 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:28:28 smithi169 systemd[1]: Starting Ceph mds.k for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:28:28.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:28 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/4128559367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:28.442 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:28 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/4128559367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:28.442 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:28:28 smithi169 podman[61190]: 2024-04-21 10:28:28.336143813 +0000 UTC m=+0.099647552 container create 58f7521f39941eb78fb655960f436114e9d460e525233d3c485fe32a6a18728e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-21T10:28:28.442 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:28:28 smithi169 podman[61190]: 2024-04-21 10:28:28.254476965 +0000 UTC m=+0.017980675 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:28:28.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:28 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/4128559367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:28.698 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:28:28 smithi169 podman[61190]: 2024-04-21 10:28:28.563780681 +0000 UTC m=+0.327284378 container init 58f7521f39941eb78fb655960f436114e9d460e525233d3c485fe32a6a18728e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-21T10:28:28.698 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:28:28 smithi169 podman[61190]: 2024-04-21 10:28:28.566441579 +0000 UTC m=+0.329945276 container start 58f7521f39941eb78fb655960f436114e9d460e525233d3c485fe32a6a18728e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:28:28.698 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:28:28 smithi169 ceph-mds[61209]: -- 172.21.15.169:0/2360544236 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55804c01da40 con 0x55804bff0800 2024-04-21T10:28:28.698 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:28:28 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-k[61205]: starting mds.k at 2024-04-21T10:28:28.698 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:28:28 smithi169 bash[61190]: 58f7521f39941eb78fb655960f436114e9d460e525233d3c485fe32a6a18728e 2024-04-21T10:28:28.698 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:28:28 smithi169 systemd[1]: Started Ceph mds.k for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:28:29.223 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.332+0000 7f86708f2640 1 -- 172.21.15.26:0/1617334492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866c1013c0 msgr2=0x7f866c1017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.332+0000 7f86708f2640 1 --2- 172.21.15.26:0/1617334492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866c1013c0 0x7f866c1017e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f865c009900 tx=0x7f865c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.332+0000 7f86708f2640 1 -- 172.21.15.26:0/1617334492 shutdown_connections 2024-04-21T10:28:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.332+0000 7f86708f2640 1 --2- 172.21.15.26:0/1617334492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f866c102a90 0x7f866c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.332+0000 7f86708f2640 1 --2- 172.21.15.26:0/1617334492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f866c1020d0 0x7f866c102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.332+0000 7f86708f2640 1 --2- 172.21.15.26:0/1617334492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866c1013c0 0x7f866c1017e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.332+0000 7f86708f2640 1 -- 172.21.15.26:0/1617334492 >> 172.21.15.26:0/1617334492 conn(0x7f866c0fbb90 msgr2=0x7f866c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.332+0000 7f86708f2640 1 -- 172.21.15.26:0/1617334492 shutdown_connections 2024-04-21T10:28:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.332+0000 7f86708f2640 1 -- 172.21.15.26:0/1617334492 wait complete. 2024-04-21T10:28:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.333+0000 7f86708f2640 1 Processor -- start 2024-04-21T10:28:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.333+0000 7f86708f2640 1 -- start start 2024-04-21T10:28:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.333+0000 7f86708f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866c1013c0 0x7f866c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.333+0000 7f86708f2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f866c1020d0 0x7f866c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.333+0000 7f86708f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f866c102a90 0x7f866c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.333+0000 7f86708f2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f866c10db30 con 0x7f866c1013c0 2024-04-21T10:28:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.333+0000 7f86708f2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f866c10d9b0 con 0x7f866c102a90 2024-04-21T10:28:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.333+0000 7f86708f2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f866c10dcb0 con 0x7f866c1020d0 2024-04-21T10:28:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.333+0000 7f866a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866c1013c0 0x7f866c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f866a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866c1013c0 0x7f866c198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44344/0 (socket says 172.21.15.26:44344) 2024-04-21T10:28:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f866a59c640 1 -- 172.21.15.26:0/607191437 learned_addr learned my addr 172.21.15.26:0/607191437 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f8669d9b640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f866c1020d0 0x7f866c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f866ad9d640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f866c102a90 0x7f866c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f866a59c640 1 -- 172.21.15.26:0/607191437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f866c1020d0 msgr2=0x7f866c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:29.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f866a59c640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f866c1020d0 0x7f866c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:29.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f866a59c640 1 -- 172.21.15.26:0/607191437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f866c102a90 msgr2=0x7f866c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:29.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f866a59c640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f866c102a90 0x7f866c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:29.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f866a59c640 1 -- 172.21.15.26:0/607191437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866c19fda0 con 0x7f866c1013c0 2024-04-21T10:28:29.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f866a59c640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866c1013c0 0x7f866c198180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f865c000c00 tx=0x7f865c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:29.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f865b7fe640 1 -- 172.21.15.26:0/607191437 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c043020 con 0x7f866c1013c0 2024-04-21T10:28:29.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f865b7fe640 1 -- 172.21.15.26:0/607191437 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f865c004440 con 0x7f866c1013c0 2024-04-21T10:28:29.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f86708f2640 1 -- 172.21.15.26:0/607191437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866c1a0030 con 0x7f866c1013c0 2024-04-21T10:28:29.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f865b7fe640 1 -- 172.21.15.26:0/607191437 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c003c90 con 0x7f866c1013c0 2024-04-21T10:28:29.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.334+0000 7f86708f2640 1 -- 172.21.15.26:0/607191437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f866c1a05d0 con 0x7f866c1013c0 2024-04-21T10:28:29.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.335+0000 7f86708f2640 1 -- 172.21.15.26:0/607191437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f866c108200 con 0x7f866c1013c0 2024-04-21T10:28:29.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.336+0000 7f865b7fe640 1 -- 172.21.15.26:0/607191437 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f865c046020 con 0x7f866c1013c0 2024-04-21T10:28:29.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.337+0000 7f865b7fe640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f86540a3c30 0x7f86540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:29.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.337+0000 7f865b7fe640 1 -- 172.21.15.26:0/607191437 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f865c0f2160 con 0x7f866c1013c0 2024-04-21T10:28:29.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.337+0000 7f8669d9b640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f86540a3c30 0x7f86540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:29.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.337+0000 7f8669d9b640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f86540a3c30 0x7f86540a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f866c199690 tx=0x7f8660005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:29.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.341+0000 7f865b7fe640 1 -- 172.21.15.26:0/607191437 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f865c04d020 con 0x7f866c1013c0 2024-04-21T10:28:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.576+0000 7f86708f2640 1 -- 172.21.15.26:0/607191437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f866c02d5c0 con 0x7f866c1013c0 2024-04-21T10:28:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.577+0000 7f865b7fe640 1 -- 172.21.15.26:0/607191437 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f865c0befe0 con 0x7f866c1013c0 2024-04-21T10:28:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:29.577 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T10:28:29.577 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 3 2024-04-21T10:28:29.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.579+0000 7f86708f2640 1 -- 172.21.15.26:0/607191437 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f86540a3c30 msgr2=0x7f86540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:29.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.579+0000 7f86708f2640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f86540a3c30 0x7f86540a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f866c199690 tx=0x7f8660005f90 comp rx=0 tx=0).stop 2024-04-21T10:28:29.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.579+0000 7f86708f2640 1 -- 172.21.15.26:0/607191437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866c1013c0 msgr2=0x7f866c198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:29.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.579+0000 7f86708f2640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866c1013c0 0x7f866c198180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f865c000c00 tx=0x7f865c004040 comp rx=0 tx=0).stop 2024-04-21T10:28:29.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.579+0000 7f86708f2640 1 -- 172.21.15.26:0/607191437 shutdown_connections 2024-04-21T10:28:29.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.579+0000 7f86708f2640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f86540a3c30 0x7f86540a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:29.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.579+0000 7f86708f2640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f866c102a90 0x7f866c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.579+0000 7f86708f2640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f866c1020d0 0x7f866c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.579+0000 7f86708f2640 1 --2- 172.21.15.26:0/607191437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866c1013c0 0x7f866c198180 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.579+0000 7f86708f2640 1 -- 172.21.15.26:0/607191437 >> 172.21.15.26:0/607191437 conn(0x7f866c0fbb90 msgr2=0x7f866c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.579+0000 7f86708f2640 1 -- 172.21.15.26:0/607191437 shutdown_connections 2024-04-21T10:28:29.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:29.579+0000 7f86708f2640 1 -- 172.21.15.26:0/607191437 wait complete. 2024-04-21T10:28:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:29 smithi026 ceph-mon[27965]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:29.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:29.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:29.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:29.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:29.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:29 smithi026 ceph-mon[27965]: Deploying daemon mds.j on smithi026 2024-04-21T10:28:29.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:29 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.169:6832/2779252400,v1:172.21.15.169:6833/2779252400] up:boot 2024-04-21T10:28:29.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:29 smithi026 ceph-mon[27965]: fsmap 2 up:standby 2024-04-21T10:28:29.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:29 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T10:28:29.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:29 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/607191437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:29.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:29 smithi169 ceph-mon[31469]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:29.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:29.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:29.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:29.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:29.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:29.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:29.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:29 smithi169 ceph-mon[31469]: Deploying daemon mds.j on smithi026 2024-04-21T10:28:29.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:29 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.169:6832/2779252400,v1:172.21.15.169:6833/2779252400] up:boot 2024-04-21T10:28:29.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:29 smithi169 ceph-mon[31469]: fsmap 2 up:standby 2024-04-21T10:28:29.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:29 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T10:28:29.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:29 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/607191437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:29 smithi190 ceph-mon[31423]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:29 smithi190 ceph-mon[31423]: Deploying daemon mds.j on smithi026 2024-04-21T10:28:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:29 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.169:6832/2779252400,v1:172.21.15.169:6833/2779252400] up:boot 2024-04-21T10:28:30.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:29 smithi190 ceph-mon[31423]: fsmap 2 up:standby 2024-04-21T10:28:30.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:29 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T10:28:30.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:29 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/607191437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:31.094 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:28:30 smithi026 systemd[1]: Starting Ceph mds.j for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:28:31.347 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:28:31 smithi026 podman[65639]: 2024-04-21 10:28:31.094565865 +0000 UTC m=+0.086851152 container create d71fb6b067d4baca0f0a039c151214409b3c098e179aadec700a77dcdea84a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-21T10:28:31.348 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:28:31 smithi026 podman[65639]: 2024-04-21 10:28:31.026100973 +0000 UTC m=+0.018386279 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:28:31.348 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:28:31 smithi026 podman[65639]: 2024-04-21 10:28:31.321948878 +0000 UTC m=+0.314234165 container init d71fb6b067d4baca0f0a039c151214409b3c098e179aadec700a77dcdea84a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:28:31.348 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:28:31 smithi026 podman[65639]: 2024-04-21 10:28:31.324687099 +0000 UTC m=+0.316972388 container start d71fb6b067d4baca0f0a039c151214409b3c098e179aadec700a77dcdea84a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:28:31.348 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:28:31 smithi026 ceph-mds[65659]: -- 172.21.15.26:0/76908747 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564a8e3e1a40 con 0x564a8e3b4c00 2024-04-21T10:28:31.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:31.613 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:28:31 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-j[65655]: starting mds.j at 2024-04-21T10:28:31.614 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:28:31 smithi026 bash[65639]: d71fb6b067d4baca0f0a039c151214409b3c098e179aadec700a77dcdea84a0b 2024-04-21T10:28:31.614 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:28:31 smithi026 systemd[1]: Started Ceph mds.j for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:28:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.698+0000 7fa75eef2640 1 -- 172.21.15.26:0/3094994718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa758100e70 msgr2=0x7fa758101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.698+0000 7fa75eef2640 1 --2- 172.21.15.26:0/3094994718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa758100e70 0x7fa758101290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009900 tx=0x7fa74c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.698+0000 7fa75eef2640 1 -- 172.21.15.26:0/3094994718 shutdown_connections 2024-04-21T10:28:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.698+0000 7fa75eef2640 1 --2- 172.21.15.26:0/3094994718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa758102980 0x7fa7581092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.698+0000 7fa75eef2640 1 --2- 172.21.15.26:0/3094994718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa758101fc0 0x7fa758102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.698+0000 7fa75eef2640 1 --2- 172.21.15.26:0/3094994718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa758100e70 0x7fa758101290 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.698+0000 7fa75eef2640 1 -- 172.21.15.26:0/3094994718 >> 172.21.15.26:0/3094994718 conn(0x7fa7580fc3e0 msgr2=0x7fa7580fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.698+0000 7fa75eef2640 1 -- 172.21.15.26:0/3094994718 shutdown_connections 2024-04-21T10:28:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.698+0000 7fa75eef2640 1 -- 172.21.15.26:0/3094994718 wait complete. 2024-04-21T10:28:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.698+0000 7fa75eef2640 1 Processor -- start 2024-04-21T10:28:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75eef2640 1 -- start start 2024-04-21T10:28:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75eef2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa758100e70 0x7fa7581981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75eef2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa758101fc0 0x7fa758198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75eef2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa758102980 0x7fa75819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75eef2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa75810c9a0 con 0x7fa758102980 2024-04-21T10:28:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75eef2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa75810c820 con 0x7fa758100e70 2024-04-21T10:28:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75eef2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa75810cb20 con 0x7fa758101fc0 2024-04-21T10:28:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75d48f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa758102980 0x7fa75819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75d48f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa758102980 0x7fa75819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44370/0 (socket says 172.21.15.26:44370) 2024-04-21T10:28:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75d48f640 1 -- 172.21.15.26:0/2236547167 learned_addr learned my addr 172.21.15.26:0/2236547167 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa757fff640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa758101fc0 0x7fa758198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75cc8e640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa758100e70 0x7fa7581981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75d48f640 1 -- 172.21.15.26:0/2236547167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa758101fc0 msgr2=0x7fa758198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75d48f640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa758101fc0 0x7fa758198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75d48f640 1 -- 172.21.15.26:0/2236547167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa758100e70 msgr2=0x7fa7581981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75d48f640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa758100e70 0x7fa7581981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75d48f640 1 -- 172.21.15.26:0/2236547167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75819fde0 con 0x7fa758102980 2024-04-21T10:28:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.699+0000 7fa75d48f640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa758102980 0x7fa75819f770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa75000eaa0 tx=0x7fa75000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:31.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.701+0000 7fa755ffb640 1 -- 172.21.15.26:0/2236547167 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75000cdb0 con 0x7fa758102980 2024-04-21T10:28:31.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.702+0000 7fa755ffb640 1 -- 172.21.15.26:0/2236547167 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa750005020 con 0x7fa758102980 2024-04-21T10:28:31.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.702+0000 7fa755ffb640 1 -- 172.21.15.26:0/2236547167 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7500116d0 con 0x7fa758102980 2024-04-21T10:28:31.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.702+0000 7fa75eef2640 1 -- 172.21.15.26:0/2236547167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7581a00d0 con 0x7fa758102980 2024-04-21T10:28:31.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.702+0000 7fa75eef2640 1 -- 172.21.15.26:0/2236547167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7581a05d0 con 0x7fa758102980 2024-04-21T10:28:31.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.703+0000 7fa75eef2640 1 -- 172.21.15.26:0/2236547167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7581a0890 con 0x7fa758102980 2024-04-21T10:28:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.708+0000 7fa755ffb640 1 -- 172.21.15.26:0/2236547167 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa7500051d0 con 0x7fa758102980 2024-04-21T10:28:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.708+0000 7fa755ffb640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa7480a3c30 0x7fa7480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:31.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.708+0000 7fa755ffb640 1 -- 172.21.15.26:0/2236547167 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fa7500d03d0 con 0x7fa758102980 2024-04-21T10:28:31.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.708+0000 7fa75cc8e640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa7480a3c30 0x7fa7480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:31.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.709+0000 7fa75cc8e640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa7480a3c30 0x7fa7480a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa74c006010 tx=0x7fa74c02c740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:31.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.712+0000 7fa755ffb640 1 -- 172.21.15.26:0/2236547167 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa750016030 con 0x7fa758102980 2024-04-21T10:28:31.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:31 smithi026 ceph-mon[27965]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:31.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:31.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:31.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:31.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:31.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:31.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:31 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:31.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.938+0000 7fa75eef2640 1 -- 172.21.15.26:0/2236547167 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7581a0a60 con 0x7fa758102980 2024-04-21T10:28:31.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.938+0000 7fa755ffb640 1 -- 172.21.15.26:0/2236547167 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fa75009cc80 con 0x7fa758102980 2024-04-21T10:28:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:31.938 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T10:28:31.938 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 4 2024-04-21T10:28:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.940+0000 7fa75eef2640 1 -- 172.21.15.26:0/2236547167 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa7480a3c30 msgr2=0x7fa7480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.940+0000 7fa75eef2640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa7480a3c30 0x7fa7480a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa74c006010 tx=0x7fa74c02c740 comp rx=0 tx=0).stop 2024-04-21T10:28:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.940+0000 7fa75eef2640 1 -- 172.21.15.26:0/2236547167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa758102980 msgr2=0x7fa75819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.940+0000 7fa75eef2640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa758102980 0x7fa75819f770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa75000eaa0 tx=0x7fa75000ef60 comp rx=0 tx=0).stop 2024-04-21T10:28:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.940+0000 7fa75eef2640 1 -- 172.21.15.26:0/2236547167 shutdown_connections 2024-04-21T10:28:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.940+0000 7fa75eef2640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa7480a3c30 0x7fa7480a60f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.940+0000 7fa75eef2640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa758102980 0x7fa75819f770 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.940+0000 7fa75eef2640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa758101fc0 0x7fa758198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.940+0000 7fa75eef2640 1 --2- 172.21.15.26:0/2236547167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa758100e70 0x7fa7581981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.940+0000 7fa75eef2640 1 -- 172.21.15.26:0/2236547167 >> 172.21.15.26:0/2236547167 conn(0x7fa7580fc3e0 msgr2=0x7fa7580fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.940+0000 7fa75eef2640 1 -- 172.21.15.26:0/2236547167 shutdown_connections 2024-04-21T10:28:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:31.940+0000 7fa75eef2640 1 -- 172.21.15.26:0/2236547167 wait complete. 2024-04-21T10:28:32.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:31 smithi190 ceph-mon[31423]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:32.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:32.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:32.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:32.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:32.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:32.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:31 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:32.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:31 smithi169 ceph-mon[31469]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:32.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:32.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:32.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:32.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:32.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:32.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:31 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:32.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:32 smithi026 ceph-mon[27965]: Deploying daemon mds.d on smithi026 2024-04-21T10:28:32.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:32 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.26:6834/2165061396,v1:172.21.15.26:6835/2165061396] up:boot 2024-04-21T10:28:32.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:32 smithi026 ceph-mon[27965]: fsmap 3 up:standby 2024-04-21T10:28:32.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:32 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T10:28:32.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:32 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2236547167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:33.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:32 smithi190 ceph-mon[31423]: Deploying daemon mds.d on smithi026 2024-04-21T10:28:33.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:32 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.26:6834/2165061396,v1:172.21.15.26:6835/2165061396] up:boot 2024-04-21T10:28:33.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:32 smithi190 ceph-mon[31423]: fsmap 3 up:standby 2024-04-21T10:28:33.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:32 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T10:28:33.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:32 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2236547167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:33.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:32 smithi169 ceph-mon[31469]: Deploying daemon mds.d on smithi026 2024-04-21T10:28:33.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:32 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.26:6834/2165061396,v1:172.21.15.26:6835/2165061396] up:boot 2024-04-21T10:28:33.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:32 smithi169 ceph-mon[31469]: fsmap 3 up:standby 2024-04-21T10:28:33.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:32 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T10:28:33.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:32 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2236547167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:33.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:33 smithi190 ceph-mon[31423]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:33.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:33 smithi026 ceph-mon[27965]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:33.936 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:28:33 smithi026 systemd[1]: Starting Ceph mds.d for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:28:33.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:34.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.059+0000 7fd091d85640 1 -- 172.21.15.26:0/4191833936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c100e70 msgr2=0x7fd08c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:34.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.059+0000 7fd091d85640 1 --2- 172.21.15.26:0/4191833936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c100e70 0x7fd08c101290 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009900 tx=0x7fd07c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.059+0000 7fd091d85640 1 -- 172.21.15.26:0/4191833936 shutdown_connections 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.059+0000 7fd091d85640 1 --2- 172.21.15.26:0/4191833936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd08c102a10 0x7fd08c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.059+0000 7fd091d85640 1 --2- 172.21.15.26:0/4191833936 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd08c102050 0x7fd08c1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.059+0000 7fd091d85640 1 --2- 172.21.15.26:0/4191833936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c100e70 0x7fd08c101290 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.059+0000 7fd091d85640 1 -- 172.21.15.26:0/4191833936 >> 172.21.15.26:0/4191833936 conn(0x7fd08c0fc470 msgr2=0x7fd08c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.059+0000 7fd091d85640 1 -- 172.21.15.26:0/4191833936 shutdown_connections 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.059+0000 7fd091d85640 1 -- 172.21.15.26:0/4191833936 wait complete. 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd091d85640 1 Processor -- start 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd091d85640 1 -- start start 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd091d85640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd08c100e70 0x7fd08c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd091d85640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd08c102050 0x7fd08c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd091d85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c102a10 0x7fd08c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd091d85640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c10c9e0 con 0x7fd08c102a10 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd091d85640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c10c860 con 0x7fd08c100e70 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd091d85640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c10cb60 con 0x7fd08c102050 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd08bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c102a10 0x7fd08c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd08bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c102a10 0x7fd08c19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44386/0 (socket says 172.21.15.26:44386) 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd08bfff640 1 -- 172.21.15.26:0/3291101000 learned_addr learned my addr 172.21.15.26:0/3291101000 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd08affd640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd08c102050 0x7fd08c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.060+0000 7fd08b7fe640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd08c100e70 0x7fd08c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd08bfff640 1 -- 172.21.15.26:0/3291101000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd08c102050 msgr2=0x7fd08c19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd08bfff640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd08c102050 0x7fd08c19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd08bfff640 1 -- 172.21.15.26:0/3291101000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd08c100e70 msgr2=0x7fd08c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd08bfff640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd08c100e70 0x7fd08c06b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd08bfff640 1 -- 172.21.15.26:0/3291101000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08c19fe00 con 0x7fd08c102a10 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd08b7fe640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd08c100e70 0x7fd08c06b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd08affd640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd08c102050 0x7fd08c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd08bfff640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c102a10 0x7fd08c19f4f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd08400eaa0 tx=0x7fd08400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd088ff9640 1 -- 172.21.15.26:0/3291101000 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08400cdb0 con 0x7fd08c102a10 2024-04-21T10:28:34.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd088ff9640 1 -- 172.21.15.26:0/3291101000 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd084005020 con 0x7fd08c102a10 2024-04-21T10:28:34.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd091d85640 1 -- 172.21.15.26:0/3291101000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08c1a0090 con 0x7fd08c102a10 2024-04-21T10:28:34.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd088ff9640 1 -- 172.21.15.26:0/3291101000 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0840116d0 con 0x7fd08c102a10 2024-04-21T10:28:34.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.061+0000 7fd091d85640 1 -- 172.21.15.26:0/3291101000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08c1a05e0 con 0x7fd08c102a10 2024-04-21T10:28:34.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.062+0000 7fd091d85640 1 -- 172.21.15.26:0/3291101000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd08c108210 con 0x7fd08c102a10 2024-04-21T10:28:34.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.063+0000 7fd088ff9640 1 -- 172.21.15.26:0/3291101000 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd0840051d0 con 0x7fd08c102a10 2024-04-21T10:28:34.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.064+0000 7fd088ff9640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd0740a3c30 0x7fd0740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:34.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.064+0000 7fd08b7fe640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd0740a3c30 0x7fd0740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:34.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.064+0000 7fd088ff9640 1 -- 172.21.15.26:0/3291101000 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fd0840d0240 con 0x7fd08c102a10 2024-04-21T10:28:34.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.064+0000 7fd08b7fe640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd0740a3c30 0x7fd0740a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0096c0 tx=0x7fd07c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:34.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.068+0000 7fd088ff9640 1 -- 172.21.15.26:0/3291101000 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd084016030 con 0x7fd08c102a10 2024-04-21T10:28:34.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:33 smithi169 ceph-mon[31469]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:34.227 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:28:33 smithi026 podman[65988]: 2024-04-21 10:28:33.981825267 +0000 UTC m=+0.093879748 container create 3675e1e3c4250ddb23e9752edfdbc46da32d3a571d2b828ba3fe99bb02224270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:28:34.228 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:28:34 smithi026 podman[65988]: 2024-04-21 10:28:33.906256849 +0000 UTC m=+0.018311336 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:28:34.228 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:28:34 smithi026 podman[65988]: 2024-04-21 10:28:34.201013948 +0000 UTC m=+0.313068416 container init 3675e1e3c4250ddb23e9752edfdbc46da32d3a571d2b828ba3fe99bb02224270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-d, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-21T10:28:34.228 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:28:34 smithi026 podman[65988]: 2024-04-21 10:28:34.204573744 +0000 UTC m=+0.316628216 container start 3675e1e3c4250ddb23e9752edfdbc46da32d3a571d2b828ba3fe99bb02224270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-d, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-21T10:28:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.301+0000 7fd091d85640 1 -- 172.21.15.26:0/3291101000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd08c062b80 con 0x7fd08c102a10 2024-04-21T10:28:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.302+0000 7fd088ff9640 1 -- 172.21.15.26:0/3291101000 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fd08409d020 con 0x7fd08c102a10 2024-04-21T10:28:34.302 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:34.302 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T10:28:34.302 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 4 2024-04-21T10:28:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.304+0000 7fd091d85640 1 -- 172.21.15.26:0/3291101000 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd0740a3c30 msgr2=0x7fd0740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.304+0000 7fd091d85640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd0740a3c30 0x7fd0740a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0096c0 tx=0x7fd07c000bc0 comp rx=0 tx=0).stop 2024-04-21T10:28:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.304+0000 7fd091d85640 1 -- 172.21.15.26:0/3291101000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c102a10 msgr2=0x7fd08c19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.304+0000 7fd091d85640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c102a10 0x7fd08c19f4f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd08400eaa0 tx=0x7fd08400ef60 comp rx=0 tx=0).stop 2024-04-21T10:28:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.304+0000 7fd091d85640 1 -- 172.21.15.26:0/3291101000 shutdown_connections 2024-04-21T10:28:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.304+0000 7fd091d85640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd0740a3c30 0x7fd0740a60f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.304+0000 7fd091d85640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c102a10 0x7fd08c19f4f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.304+0000 7fd091d85640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd08c102050 0x7fd08c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.304+0000 7fd091d85640 1 --2- 172.21.15.26:0/3291101000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd08c100e70 0x7fd08c06b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.304+0000 7fd091d85640 1 -- 172.21.15.26:0/3291101000 >> 172.21.15.26:0/3291101000 conn(0x7fd08c0fc470 msgr2=0x7fd08c1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.305+0000 7fd091d85640 1 -- 172.21.15.26:0/3291101000 shutdown_connections 2024-04-21T10:28:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:34.305+0000 7fd091d85640 1 -- 172.21.15.26:0/3291101000 wait complete. 2024-04-21T10:28:34.685 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:28:34 smithi026 ceph-mds[66046]: -- 172.21.15.26:0/405081200 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55abdc67fa40 con 0x55abdc652c00 2024-04-21T10:28:34.685 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:28:34 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-d[66042]: starting mds.d at 2024-04-21T10:28:34.685 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:28:34 smithi026 bash[65988]: 3675e1e3c4250ddb23e9752edfdbc46da32d3a571d2b828ba3fe99bb02224270 2024-04-21T10:28:34.685 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:28:34 smithi026 systemd[1]: Started Ceph mds.d for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:28:35.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:34 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3291101000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:35.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:34 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:35.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:34 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:35.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:34 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:35.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:34 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:35.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:34 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:35.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:34 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:35.030 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:34 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3291101000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:35.030 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:34 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:35.030 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:34 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:35.030 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:34 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:35.030 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:34 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:35.031 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:34 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:35.031 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:34 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:35.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:34 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3291101000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:35.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:35.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:35.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:35.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:35.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:35.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:34 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:36.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:35 smithi190 ceph-mon[31423]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:36.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:35 smithi190 ceph-mon[31423]: Deploying daemon mds.e on smithi169 2024-04-21T10:28:36.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:35 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.26:6836/4049177866,v1:172.21.15.26:6837/4049177866] up:boot 2024-04-21T10:28:36.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:35 smithi190 ceph-mon[31423]: fsmap 4 up:standby 2024-04-21T10:28:36.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:35 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T10:28:36.060 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:35 smithi169 ceph-mon[31469]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:36.060 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:35 smithi169 ceph-mon[31469]: Deploying daemon mds.e on smithi169 2024-04-21T10:28:36.060 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:35 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.26:6836/4049177866,v1:172.21.15.26:6837/4049177866] up:boot 2024-04-21T10:28:36.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:35 smithi169 ceph-mon[31469]: fsmap 4 up:standby 2024-04-21T10:28:36.061 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:35 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T10:28:36.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:35 smithi026 ceph-mon[27965]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:36.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:35 smithi026 ceph-mon[27965]: Deploying daemon mds.e on smithi169 2024-04-21T10:28:36.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:35 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.26:6836/4049177866,v1:172.21.15.26:6837/4049177866] up:boot 2024-04-21T10:28:36.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:35 smithi026 ceph-mon[27965]: fsmap 4 up:standby 2024-04-21T10:28:36.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:35 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T10:28:36.317 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:36.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.423+0000 7ff2bb693640 1 -- 172.21.15.26:0/591399179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2b4100e70 msgr2=0x7ff2b4101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.423+0000 7ff2bb693640 1 --2- 172.21.15.26:0/591399179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2b4100e70 0x7ff2b4101290 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009900 tx=0x7ff2b002bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.423+0000 7ff2bb693640 1 -- 172.21.15.26:0/591399179 shutdown_connections 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.423+0000 7ff2bb693640 1 --2- 172.21.15.26:0/591399179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b4102980 0x7ff2b41092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.423+0000 7ff2bb693640 1 --2- 172.21.15.26:0/591399179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2b4101fc0 0x7ff2b4102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.423+0000 7ff2bb693640 1 --2- 172.21.15.26:0/591399179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2b4100e70 0x7ff2b4101290 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.423+0000 7ff2bb693640 1 -- 172.21.15.26:0/591399179 >> 172.21.15.26:0/591399179 conn(0x7ff2b40fc3e0 msgr2=0x7ff2b40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.424+0000 7ff2bb693640 1 -- 172.21.15.26:0/591399179 shutdown_connections 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.424+0000 7ff2bb693640 1 -- 172.21.15.26:0/591399179 wait complete. 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.424+0000 7ff2bb693640 1 Processor -- start 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.424+0000 7ff2bb693640 1 -- start start 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.424+0000 7ff2bb693640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b4100e70 0x7ff2b41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.424+0000 7ff2bb693640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2b4101fc0 0x7ff2b4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:36.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.424+0000 7ff2bb693640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2b4102980 0x7ff2b419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.424+0000 7ff2bb693640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b410c9a0 con 0x7ff2b4101fc0 2024-04-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.424+0000 7ff2bb693640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b410c820 con 0x7ff2b4102980 2024-04-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.424+0000 7ff2bb693640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b410cb20 con 0x7ff2b4100e70 2024-04-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b8c2e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2b4101fc0 0x7ff2b4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b8c2e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2b4101fc0 0x7ff2b4198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38984/0 (socket says 172.21.15.26:38984) 2024-04-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b8c2e640 1 -- 172.21.15.26:0/3994519949 learned_addr learned my addr 172.21.15.26:0/3994519949 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b9c30640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2b4102980 0x7ff2b419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b942f640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b4100e70 0x7ff2b41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b8c2e640 1 -- 172.21.15.26:0/3994519949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b4100e70 msgr2=0x7ff2b41981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:36.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b8c2e640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b4100e70 0x7ff2b41981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:36.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b8c2e640 1 -- 172.21.15.26:0/3994519949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2b4102980 msgr2=0x7ff2b419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:36.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b8c2e640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2b4102980 0x7ff2b419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:36.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b8c2e640 1 -- 172.21.15.26:0/3994519949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b419fde0 con 0x7ff2b4101fc0 2024-04-21T10:28:36.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b942f640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b4100e70 0x7ff2b41981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:28:36.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b9c30640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2b4102980 0x7ff2b419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:28:36.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2b8c2e640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2b4101fc0 0x7ff2b4198700 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400e9c0 tx=0x7ff2a400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:36.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2aa7fc640 1 -- 172.21.15.26:0/3994519949 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400cde0 con 0x7ff2b4101fc0 2024-04-21T10:28:36.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2aa7fc640 1 -- 172.21.15.26:0/3994519949 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2a40108e0 con 0x7ff2b4101fc0 2024-04-21T10:28:36.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2bb693640 1 -- 172.21.15.26:0/3994519949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b41a00d0 con 0x7ff2b4101fc0 2024-04-21T10:28:36.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2aa7fc640 1 -- 172.21.15.26:0/3994519949 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4018960 con 0x7ff2b4101fc0 2024-04-21T10:28:36.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.425+0000 7ff2bb693640 1 -- 172.21.15.26:0/3994519949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b41a0620 con 0x7ff2b4101fc0 2024-04-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.427+0000 7ff2bb693640 1 -- 172.21.15.26:0/3994519949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b41a0880 con 0x7ff2b4101fc0 2024-04-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.427+0000 7ff2aa7fc640 1 -- 172.21.15.26:0/3994519949 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff2a401f020 con 0x7ff2b4101fc0 2024-04-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.428+0000 7ff2aa7fc640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff2a00a3bb0 0x7ff2a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.428+0000 7ff2aa7fc640 1 -- 172.21.15.26:0/3994519949 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ff2a40d0280 con 0x7ff2b4101fc0 2024-04-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.428+0000 7ff2b942f640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff2a00a3bb0 0x7ff2a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:36.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.429+0000 7ff2b942f640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff2a00a3bb0 0x7ff2a00a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0006010 tx=0x7ff2b002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:36.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.433+0000 7ff2aa7fc640 1 -- 172.21.15.26:0/3994519949 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2a4053ff0 con 0x7ff2b4101fc0 2024-04-21T10:28:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.680+0000 7ff2bb693640 1 -- 172.21.15.26:0/3994519949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2b41a0a50 con 0x7ff2b4101fc0 2024-04-21T10:28:36.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.681+0000 7ff2aa7fc640 1 -- 172.21.15.26:0/3994519949 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7ff2a409d100 con 0x7ff2b4101fc0 2024-04-21T10:28:36.681 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:36.681 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T10:28:36.681 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 5 2024-04-21T10:28:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.683+0000 7ff2bb693640 1 -- 172.21.15.26:0/3994519949 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff2a00a3bb0 msgr2=0x7ff2a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.683+0000 7ff2bb693640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff2a00a3bb0 0x7ff2a00a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0006010 tx=0x7ff2b002c7e0 comp rx=0 tx=0).stop 2024-04-21T10:28:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.683+0000 7ff2bb693640 1 -- 172.21.15.26:0/3994519949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2b4101fc0 msgr2=0x7ff2b4198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.683+0000 7ff2bb693640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2b4101fc0 0x7ff2b4198700 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400e9c0 tx=0x7ff2a400ee80 comp rx=0 tx=0).stop 2024-04-21T10:28:36.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.683+0000 7ff2bb693640 1 -- 172.21.15.26:0/3994519949 shutdown_connections 2024-04-21T10:28:36.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.683+0000 7ff2bb693640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff2a00a3bb0 0x7ff2a00a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:36.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.683+0000 7ff2bb693640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2b4102980 0x7ff2b419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:36.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.683+0000 7ff2bb693640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2b4101fc0 0x7ff2b4198700 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:36.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.683+0000 7ff2bb693640 1 --2- 172.21.15.26:0/3994519949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2b4100e70 0x7ff2b41981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:36.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.683+0000 7ff2bb693640 1 -- 172.21.15.26:0/3994519949 >> 172.21.15.26:0/3994519949 conn(0x7ff2b40fc3e0 msgr2=0x7ff2b40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:36.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.683+0000 7ff2bb693640 1 -- 172.21.15.26:0/3994519949 shutdown_connections 2024-04-21T10:28:36.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:36.683+0000 7ff2bb693640 1 -- 172.21.15.26:0/3994519949 wait complete. 2024-04-21T10:28:36.698 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:28:36 smithi169 systemd[1]: Starting Ceph mds.e for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:28:36.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:36 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3994519949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:36.982 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:36 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3994519949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:36.983 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:28:36 smithi169 podman[61496]: 2024-04-21 10:28:36.64804871 +0000 UTC m=+0.017931545 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:28:36.983 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:28:36 smithi169 podman[61496]: 2024-04-21 10:28:36.756251699 +0000 UTC m=+0.126134574 container create ed196f30de18d8c8d3b9b1abdc55ab729674b87b61dd6c21924ec1b74dbff23f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:28:36.983 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:28:36 smithi169 podman[61496]: 2024-04-21 10:28:36.958414051 +0000 UTC m=+0.328296882 container init ed196f30de18d8c8d3b9b1abdc55ab729674b87b61dd6c21924ec1b74dbff23f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:28:36.983 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:28:36 smithi169 podman[61496]: 2024-04-21 10:28:36.961091535 +0000 UTC m=+0.330974368 container start ed196f30de18d8c8d3b9b1abdc55ab729674b87b61dd6c21924ec1b74dbff23f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-21T10:28:36.983 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:28:36 smithi169 ceph-mds[61515]: -- 172.21.15.169:0/1250291116 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570866dfa40 con 0x5570866b2c00 2024-04-21T10:28:37.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:36 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3994519949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:37.448 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:28:36 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e[61511]: starting mds.e at 2024-04-21T10:28:37.448 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:28:37 smithi169 bash[61496]: ed196f30de18d8c8d3b9b1abdc55ab729674b87b61dd6c21924ec1b74dbff23f 2024-04-21T10:28:37.448 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:28:37 smithi169 systemd[1]: Started Ceph mds.e for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:28:38.021 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:37 smithi190 ceph-mon[31423]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:38.021 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:38.021 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:38.021 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:38.021 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:38.021 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:38.021 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:37 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:38.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:37 smithi026 ceph-mon[27965]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:38.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:38.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:38.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:38.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:38.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:38.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:37 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:38.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:37 smithi169 ceph-mon[31469]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:38.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:38.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:38.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:38.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:38.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:38.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:37 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:38.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:38.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.812+0000 7f3698281640 1 -- 172.21.15.26:0/2271602046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3690105380 msgr2=0x7f3690107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:38.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.812+0000 7f3698281640 1 --2- 172.21.15.26:0/2271602046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3690105380 0x7f3690107770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3680009900 tx=0x7f368002bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 -- 172.21.15.26:0/2271602046 shutdown_connections 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 --2- 172.21.15.26:0/2271602046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3690107cb0 0x7f369010a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 --2- 172.21.15.26:0/2271602046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3690105380 0x7f3690107770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 --2- 172.21.15.26:0/2271602046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3690102a50 0x7f3690104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 -- 172.21.15.26:0/2271602046 >> 172.21.15.26:0/2271602046 conn(0x7f36900fc420 msgr2=0x7f36900fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 -- 172.21.15.26:0/2271602046 shutdown_connections 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 -- 172.21.15.26:0/2271602046 wait complete. 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 Processor -- start 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 -- start start 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3690102a50 0x7f36901981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3690105380 0x7f3690198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3690107cb0 0x7f369019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f369010daf0 con 0x7f3690102a50 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f369010d970 con 0x7f3690107cb0 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f3698281640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f369010dc70 con 0x7f3690105380 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.813+0000 7f369601d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3690102a50 0x7f36901981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f369601d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3690102a50 0x7f36901981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39000/0 (socket says 172.21.15.26:39000) 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f369601d640 1 -- 172.21.15.26:0/2862799205 learned_addr learned my addr 172.21.15.26:0/2862799205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f369581c640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3690105380 0x7f3690198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f369681e640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3690107cb0 0x7f369019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f369601d640 1 -- 172.21.15.26:0/2862799205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3690105380 msgr2=0x7f3690198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f369601d640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3690105380 0x7f3690198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f369601d640 1 -- 172.21.15.26:0/2862799205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3690107cb0 msgr2=0x7f369019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f369601d640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3690107cb0 0x7f369019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f369601d640 1 -- 172.21.15.26:0/2862799205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369019fde0 con 0x7f3690102a50 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f369601d640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3690102a50 0x7f36901981c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f368c00c920 tx=0x7f368c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:38.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f36877fe640 1 -- 172.21.15.26:0/2862799205 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c007c70 con 0x7f3690102a50 2024-04-21T10:28:38.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f36877fe640 1 -- 172.21.15.26:0/2862799205 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f368c007e10 con 0x7f3690102a50 2024-04-21T10:28:38.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f36877fe640 1 -- 172.21.15.26:0/2862799205 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c018820 con 0x7f3690102a50 2024-04-21T10:28:38.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f3698281640 1 -- 172.21.15.26:0/2862799205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36901a00d0 con 0x7f3690102a50 2024-04-21T10:28:38.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.814+0000 7f3698281640 1 -- 172.21.15.26:0/2862799205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36901a0640 con 0x7f3690102a50 2024-04-21T10:28:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.815+0000 7f3698281640 1 -- 172.21.15.26:0/2862799205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36900fdf10 con 0x7f3690102a50 2024-04-21T10:28:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.816+0000 7f36877fe640 1 -- 172.21.15.26:0/2862799205 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f368c00f450 con 0x7f3690102a50 2024-04-21T10:28:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.817+0000 7f36877fe640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f367c0a3c30 0x7f367c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.817+0000 7f369581c640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f367c0a3c30 0x7f367c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.818+0000 7f36877fe640 1 -- 172.21.15.26:0/2862799205 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f368c0d0010 con 0x7f3690102a50 2024-04-21T10:28:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.818+0000 7f369581c640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f367c0a3c30 0x7f367c0a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f368002c240 tx=0x7f368002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:38.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:38.822+0000 7f36877fe640 1 -- 172.21.15.26:0/2862799205 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f368c014030 con 0x7f3690102a50 2024-04-21T10:28:38.930 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:38 smithi190 ceph-mon[31423]: Deploying daemon mds.f on smithi190 2024-04-21T10:28:38.930 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:38 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.169:6834/1594119566,v1:172.21.15.169:6835/1594119566] up:boot 2024-04-21T10:28:38.930 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:38 smithi190 ceph-mon[31423]: fsmap 5 up:standby 2024-04-21T10:28:38.930 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:38 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T10:28:39.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.047+0000 7f3698281640 1 -- 172.21.15.26:0/2862799205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f369002d5c0 con 0x7f3690102a50 2024-04-21T10:28:39.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.048+0000 7f36877fe640 1 -- 172.21.15.26:0/2862799205 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7f368c09ce90 con 0x7f3690102a50 2024-04-21T10:28:39.048 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:39.048 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-21T10:28:39.048 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 6 2024-04-21T10:28:39.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.050+0000 7f3698281640 1 -- 172.21.15.26:0/2862799205 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f367c0a3c30 msgr2=0x7f367c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.050+0000 7f3698281640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f367c0a3c30 0x7f367c0a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f368002c240 tx=0x7f368002c7e0 comp rx=0 tx=0).stop 2024-04-21T10:28:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.050+0000 7f3698281640 1 -- 172.21.15.26:0/2862799205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3690102a50 msgr2=0x7f36901981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.050+0000 7f3698281640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3690102a50 0x7f36901981c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f368c00c920 tx=0x7f368c00cde0 comp rx=0 tx=0).stop 2024-04-21T10:28:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.050+0000 7f3698281640 1 -- 172.21.15.26:0/2862799205 shutdown_connections 2024-04-21T10:28:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.050+0000 7f3698281640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f367c0a3c30 0x7f367c0a60f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.050+0000 7f3698281640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3690107cb0 0x7f369019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.050+0000 7f3698281640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3690105380 0x7f3690198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.050+0000 7f3698281640 1 --2- 172.21.15.26:0/2862799205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3690102a50 0x7f36901981c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.050+0000 7f3698281640 1 -- 172.21.15.26:0/2862799205 >> 172.21.15.26:0/2862799205 conn(0x7f36900fc420 msgr2=0x7f36901033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.050+0000 7f3698281640 1 -- 172.21.15.26:0/2862799205 shutdown_connections 2024-04-21T10:28:39.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:39.050+0000 7f3698281640 1 -- 172.21.15.26:0/2862799205 wait complete. 2024-04-21T10:28:39.059 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:38 smithi026 ceph-mon[27965]: Deploying daemon mds.f on smithi190 2024-04-21T10:28:39.059 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:38 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.169:6834/1594119566,v1:172.21.15.169:6835/1594119566] up:boot 2024-04-21T10:28:39.059 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:38 smithi026 ceph-mon[27965]: fsmap 5 up:standby 2024-04-21T10:28:39.059 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:38 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T10:28:39.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:38 smithi169 ceph-mon[31469]: Deploying daemon mds.f on smithi190 2024-04-21T10:28:39.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:38 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.169:6834/1594119566,v1:172.21.15.169:6835/1594119566] up:boot 2024-04-21T10:28:39.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:38 smithi169 ceph-mon[31469]: fsmap 5 up:standby 2024-04-21T10:28:39.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:38 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T10:28:39.277 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:28:39 smithi190 systemd[1]: Starting Ceph mds.f for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:28:39.579 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:28:39 smithi190 podman[62109]: 2024-04-21 10:28:39.223677382 +0000 UTC m=+0.017988537 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:28:39.579 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:28:39 smithi190 podman[62109]: 2024-04-21 10:28:39.327619994 +0000 UTC m=+0.121931208 container create 2a1ab1cd00df3d1f71b56395747e4908593e8873db2d203e68bcdbf926b01ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:28:39.579 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:28:39 smithi190 podman[62109]: 2024-04-21 10:28:39.580253341 +0000 UTC m=+0.374564496 container init 2a1ab1cd00df3d1f71b56395747e4908593e8873db2d203e68bcdbf926b01ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-04-21T10:28:40.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:39 smithi190 ceph-mon[31423]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:40.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:39 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2862799205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:40.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:39 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:40.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:39 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:40.027 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:28:39 smithi190 podman[62109]: 2024-04-21 10:28:39.58301107 +0000 UTC m=+0.377322234 container start 2a1ab1cd00df3d1f71b56395747e4908593e8873db2d203e68bcdbf926b01ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-21T10:28:40.027 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:28:39 smithi190 ceph-mds[62128]: -- 172.21.15.190:0/477708221 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563f5a427a40 con 0x563f5a3fac00 2024-04-21T10:28:40.027 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:28:39 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-f[62124]: starting mds.f at 2024-04-21T10:28:40.027 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:28:39 smithi190 bash[62109]: 2a1ab1cd00df3d1f71b56395747e4908593e8873db2d203e68bcdbf926b01ba0 2024-04-21T10:28:40.027 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:28:39 smithi190 systemd[1]: Started Ceph mds.f for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:28:40.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:39 smithi026 ceph-mon[27965]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:40.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:39 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2862799205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:40.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:39 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:40.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:39 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:40.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:39 smithi169 ceph-mon[31469]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:40.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:39 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2862799205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:40.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:39 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:40.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:39 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:41.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:40 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.190:6834/1573483809,v1:172.21.15.190:6835/1573483809] up:boot 2024-04-21T10:28:41.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:40 smithi190 ceph-mon[31423]: fsmap 6 up:standby 2024-04-21T10:28:41.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T10:28:41.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:41.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:41.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:41.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:40 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:41.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:40 smithi190 ceph-mon[31423]: Deploying daemon mds.c on smithi190 2024-04-21T10:28:41.061 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:41.086 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:40 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.190:6834/1573483809,v1:172.21.15.190:6835/1573483809] up:boot 2024-04-21T10:28:41.087 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:40 smithi026 ceph-mon[27965]: fsmap 6 up:standby 2024-04-21T10:28:41.087 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T10:28:41.087 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:41.087 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:41.087 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:41.087 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:40 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:41.087 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:40 smithi026 ceph-mon[27965]: Deploying daemon mds.c on smithi190 2024-04-21T10:28:41.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.175+0000 7fd150ec1640 1 -- 172.21.15.26:0/759821983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c100890 msgr2=0x7fd14c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:41.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.175+0000 7fd150ec1640 1 --2- 172.21.15.26:0/759821983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c100890 0x7fd14c100cb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009900 tx=0x7fd13c02bcd0 comp rx=0 tx=0).stop 2024-04-21T10:28:41.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.175+0000 7fd150ec1640 1 -- 172.21.15.26:0/759821983 shutdown_connections 2024-04-21T10:28:41.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.175+0000 7fd150ec1640 1 --2- 172.21.15.26:0/759821983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd14c101b90 0x7fd14c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:41.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.175+0000 7fd150ec1640 1 --2- 172.21.15.26:0/759821983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd14c1011f0 0x7fd14c101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:41.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.175+0000 7fd150ec1640 1 --2- 172.21.15.26:0/759821983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c100890 0x7fd14c100cb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:41.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.175+0000 7fd150ec1640 1 -- 172.21.15.26:0/759821983 >> 172.21.15.26:0/759821983 conn(0x7fd14c0fc410 msgr2=0x7fd14c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:41.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.175+0000 7fd150ec1640 1 -- 172.21.15.26:0/759821983 shutdown_connections 2024-04-21T10:28:41.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.175+0000 7fd150ec1640 1 -- 172.21.15.26:0/759821983 wait complete. 2024-04-21T10:28:41.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.176+0000 7fd150ec1640 1 Processor -- start 2024-04-21T10:28:41.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.176+0000 7fd150ec1640 1 -- start start 2024-04-21T10:28:41.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.176+0000 7fd150ec1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd14c100890 0x7fd14c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:41.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.176+0000 7fd150ec1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1011f0 0x7fd14c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:41.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.176+0000 7fd150ec1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd14c101b90 0x7fd14c1a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:41.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.176+0000 7fd150ec1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c10fce0 con 0x7fd14c1011f0 2024-04-21T10:28:41.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.176+0000 7fd150ec1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c10fb60 con 0x7fd14c100890 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.176+0000 7fd150ec1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c10fe60 con 0x7fd14c101b90 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.176+0000 7fd149d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1011f0 0x7fd14c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.176+0000 7fd149d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1011f0 0x7fd14c19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39024/0 (socket says 172.21.15.26:39024) 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.176+0000 7fd149d9b640 1 -- 172.21.15.26:0/1068496486 learned_addr learned my addr 172.21.15.26:0/1068496486 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.176+0000 7fd14a59c640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd14c100890 0x7fd14c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.177+0000 7fd14ad9d640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd14c101b90 0x7fd14c1a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.177+0000 7fd149d9b640 1 -- 172.21.15.26:0/1068496486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd14c101b90 msgr2=0x7fd14c1a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.177+0000 7fd149d9b640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd14c101b90 0x7fd14c1a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.177+0000 7fd149d9b640 1 -- 172.21.15.26:0/1068496486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd14c100890 msgr2=0x7fd14c19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.177+0000 7fd149d9b640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd14c100890 0x7fd14c19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.177+0000 7fd149d9b640 1 -- 172.21.15.26:0/1068496486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14c1a1fb0 con 0x7fd14c1011f0 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.177+0000 7fd149d9b640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1011f0 0x7fd14c19a8d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c890 tx=0x7fd14000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.177+0000 7fd13b7fe640 1 -- 172.21.15.26:0/1068496486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140004220 con 0x7fd14c1011f0 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.177+0000 7fd13b7fe640 1 -- 172.21.15.26:0/1068496486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1400043c0 con 0x7fd14c1011f0 2024-04-21T10:28:41.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.177+0000 7fd150ec1640 1 -- 172.21.15.26:0/1068496486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14c1a22a0 con 0x7fd14c1011f0 2024-04-21T10:28:41.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.177+0000 7fd13b7fe640 1 -- 172.21.15.26:0/1068496486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140004a00 con 0x7fd14c1011f0 2024-04-21T10:28:41.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.177+0000 7fd150ec1640 1 -- 172.21.15.26:0/1068496486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14c1a27f0 con 0x7fd14c1011f0 2024-04-21T10:28:41.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.178+0000 7fd150ec1640 1 -- 172.21.15.26:0/1068496486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd14c10b770 con 0x7fd14c1011f0 2024-04-21T10:28:41.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.179+0000 7fd13b7fe640 1 -- 172.21.15.26:0/1068496486 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd1400079b0 con 0x7fd14c1011f0 2024-04-21T10:28:41.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.180+0000 7fd13b7fe640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd1340a3bb0 0x7fd1340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:41.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.180+0000 7fd14a59c640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd1340a3bb0 0x7fd1340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:41.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.180+0000 7fd13b7fe640 1 -- 172.21.15.26:0/1068496486 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fd1400cfc90 con 0x7fd14c1011f0 2024-04-21T10:28:41.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.181+0000 7fd14a59c640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd1340a3bb0 0x7fd1340a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009230 tx=0x7fd13c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:41.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.185+0000 7fd13b7fe640 1 -- 172.21.15.26:0/1068496486 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd140014030 con 0x7fd14c1011f0 2024-04-21T10:28:41.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:40 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.190:6834/1573483809,v1:172.21.15.190:6835/1573483809] up:boot 2024-04-21T10:28:41.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:40 smithi169 ceph-mon[31469]: fsmap 6 up:standby 2024-04-21T10:28:41.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T10:28:41.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:41.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:41.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:41.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:40 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:41.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:40 smithi169 ceph-mon[31469]: Deploying daemon mds.c on smithi190 2024-04-21T10:28:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.412+0000 7fd150ec1640 1 -- 172.21.15.26:0/1068496486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd14c02d5c0 con 0x7fd14c1011f0 2024-04-21T10:28:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.413+0000 7fd13b7fe640 1 -- 172.21.15.26:0/1068496486 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7fd14009c420 con 0x7fd14c1011f0 2024-04-21T10:28:41.413 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:41.413 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-21T10:28:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-04-21T10:28:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.415+0000 7fd150ec1640 1 -- 172.21.15.26:0/1068496486 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd1340a3bb0 msgr2=0x7fd1340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.415+0000 7fd150ec1640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd1340a3bb0 0x7fd1340a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009230 tx=0x7fd13c005fd0 comp rx=0 tx=0).stop 2024-04-21T10:28:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.415+0000 7fd150ec1640 1 -- 172.21.15.26:0/1068496486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1011f0 msgr2=0x7fd14c19a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.415+0000 7fd150ec1640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1011f0 0x7fd14c19a8d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c890 tx=0x7fd14000cd50 comp rx=0 tx=0).stop 2024-04-21T10:28:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.415+0000 7fd150ec1640 1 -- 172.21.15.26:0/1068496486 shutdown_connections 2024-04-21T10:28:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.415+0000 7fd150ec1640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd1340a3bb0 0x7fd1340a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.415+0000 7fd150ec1640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd14c101b90 0x7fd14c1a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.415+0000 7fd150ec1640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd14c1011f0 0x7fd14c19a8d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.415+0000 7fd150ec1640 1 --2- 172.21.15.26:0/1068496486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd14c100890 0x7fd14c19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.415+0000 7fd150ec1640 1 -- 172.21.15.26:0/1068496486 >> 172.21.15.26:0/1068496486 conn(0x7fd14c0fc410 msgr2=0x7fd14c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.415+0000 7fd150ec1640 1 -- 172.21.15.26:0/1068496486 shutdown_connections 2024-04-21T10:28:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:41.415+0000 7fd150ec1640 1 -- 172.21.15.26:0/1068496486 wait complete. 2024-04-21T10:28:42.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:41 smithi190 ceph-mon[31423]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:42.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:41 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1068496486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:42.027 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:28:41 smithi190 systemd[1]: Starting Ceph mds.c for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:28:42.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:41 smithi026 ceph-mon[27965]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:42.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:41 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1068496486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:42.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:41 smithi169 ceph-mon[31469]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:42.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:41 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1068496486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:42.457 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:28:42 smithi190 podman[62415]: 2024-04-21 10:28:42.080874347 +0000 UTC m=+0.017944248 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:28:42.457 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:28:42 smithi190 podman[62415]: 2024-04-21 10:28:42.19716355 +0000 UTC m=+0.134233436 container create 945f9b9fd7f44de64a4ecf7e3fd56443c82af5178da400cd8fcab55d258a6f64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-21T10:28:42.457 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:28:42 smithi190 podman[62415]: 2024-04-21 10:28:42.457942384 +0000 UTC m=+0.395012269 container init 945f9b9fd7f44de64a4ecf7e3fd56443c82af5178da400cd8fcab55d258a6f64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-21T10:28:42.777 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:28:42 smithi190 podman[62415]: 2024-04-21 10:28:42.460595063 +0000 UTC m=+0.397664959 container start 945f9b9fd7f44de64a4ecf7e3fd56443c82af5178da400cd8fcab55d258a6f64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-21T10:28:42.777 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:28:42 smithi190 ceph-mds[62433]: -- 172.21.15.190:0/2299224233 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c00290da40 con 0x55c0028e0800 2024-04-21T10:28:42.777 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:28:42 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c[62429]: starting mds.c at 2024-04-21T10:28:42.777 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:28:42 smithi190 bash[62415]: 945f9b9fd7f44de64a4ecf7e3fd56443c82af5178da400cd8fcab55d258a6f64 2024-04-21T10:28:42.777 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:28:42 smithi190 systemd[1]: Started Ceph mds.c for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:28:43.426 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:43.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.541+0000 7f169ebe6640 1 -- 172.21.15.26:0/1986394564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1698103190 msgr2=0x7f1698103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.541+0000 7f169ebe6640 1 --2- 172.21.15.26:0/1986394564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1698103190 0x7f1698103610 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f168c009900 tx=0x7f168c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.541+0000 7f169ebe6640 1 -- 172.21.15.26:0/1986394564 shutdown_connections 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.541+0000 7f169ebe6640 1 --2- 172.21.15.26:0/1986394564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1698103b50 0x7f169810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.541+0000 7f169ebe6640 1 --2- 172.21.15.26:0/1986394564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1698103190 0x7f1698103610 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.541+0000 7f169ebe6640 1 --2- 172.21.15.26:0/1986394564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1698101fa0 0x7f16981023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.541+0000 7f169ebe6640 1 -- 172.21.15.26:0/1986394564 >> 172.21.15.26:0/1986394564 conn(0x7f16980fd530 msgr2=0x7f16980ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.541+0000 7f169ebe6640 1 -- 172.21.15.26:0/1986394564 shutdown_connections 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.541+0000 7f169ebe6640 1 -- 172.21.15.26:0/1986394564 wait complete. 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f169ebe6640 1 Processor -- start 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f169ebe6640 1 -- start start 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f169ebe6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1698101fa0 0x7f16981980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f169ebe6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1698103190 0x7f1698198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:43.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f169ebe6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1698103b50 0x7f169819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f169ebe6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f169810dae0 con 0x7f1698103190 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f169ebe6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f169810d960 con 0x7f1698101fa0 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f169ebe6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f169810dc60 con 0x7f1698103b50 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f1697fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1698103190 0x7f1698198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f1697fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1698103190 0x7f1698198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39046/0 (socket says 172.21.15.26:39046) 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f1697fff640 1 -- 172.21.15.26:0/2681015871 learned_addr learned my addr 172.21.15.26:0/2681015871 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f169c982640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1698101fa0 0x7f16981980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f169d183640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1698103b50 0x7f169819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.542+0000 7f1697fff640 1 -- 172.21.15.26:0/2681015871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1698103b50 msgr2=0x7f169819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.543+0000 7f1697fff640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1698103b50 0x7f169819f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.543+0000 7f1697fff640 1 -- 172.21.15.26:0/2681015871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1698101fa0 msgr2=0x7f16981980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.543+0000 7f1697fff640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1698101fa0 0x7f16981980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.543+0000 7f1697fff640 1 -- 172.21.15.26:0/2681015871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169819fd00 con 0x7f1698103190 2024-04-21T10:28:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.543+0000 7f169c982640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1698101fa0 0x7f16981980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:28:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.543+0000 7f1697fff640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1698103190 0x7f1698198620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f168c006010 tx=0x7f168c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.543+0000 7f1695ffb640 1 -- 172.21.15.26:0/2681015871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c043020 con 0x7f1698103190 2024-04-21T10:28:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.543+0000 7f1695ffb640 1 -- 172.21.15.26:0/2681015871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f168c004440 con 0x7f1698103190 2024-04-21T10:28:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.543+0000 7f169ebe6640 1 -- 172.21.15.26:0/2681015871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169819ff90 con 0x7f1698103190 2024-04-21T10:28:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.543+0000 7f1695ffb640 1 -- 172.21.15.26:0/2681015871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c003bf0 con 0x7f1698103190 2024-04-21T10:28:43.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.543+0000 7f169ebe6640 1 -- 172.21.15.26:0/2681015871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16981a0450 con 0x7f1698103190 2024-04-21T10:28:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.544+0000 7f169ebe6640 1 -- 172.21.15.26:0/2681015871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1698109260 con 0x7f1698103190 2024-04-21T10:28:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.545+0000 7f1695ffb640 1 -- 172.21.15.26:0/2681015871 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f168c046020 con 0x7f1698103190 2024-04-21T10:28:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.546+0000 7f1695ffb640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f16800a3bb0 0x7f16800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.546+0000 7f1695ffb640 1 -- 172.21.15.26:0/2681015871 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f168c0f29d0 con 0x7f1698103190 2024-04-21T10:28:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.546+0000 7f169c982640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f16800a3bb0 0x7f16800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:43.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.547+0000 7f169c982640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f16800a3bb0 0x7f16800a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1698102ff0 tx=0x7f1690007870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.550+0000 7f1695ffb640 1 -- 172.21.15.26:0/2681015871 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f168c04d020 con 0x7f1698103190 2024-04-21T10:28:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.777+0000 7f169ebe6640 1 -- 172.21.15.26:0/2681015871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f169802d5c0 con 0x7f1698103190 2024-04-21T10:28:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.778+0000 7f1695ffb640 1 -- 172.21.15.26:0/2681015871 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7f168c0bf7b0 con 0x7f1698103190 2024-04-21T10:28:43.778 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:43.778 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-21T10:28:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-04-21T10:28:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.780+0000 7f169ebe6640 1 -- 172.21.15.26:0/2681015871 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f16800a3bb0 msgr2=0x7f16800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.780+0000 7f169ebe6640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f16800a3bb0 0x7f16800a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1698102ff0 tx=0x7f1690007870 comp rx=0 tx=0).stop 2024-04-21T10:28:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.780+0000 7f169ebe6640 1 -- 172.21.15.26:0/2681015871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1698103190 msgr2=0x7f1698198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.780+0000 7f169ebe6640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1698103190 0x7f1698198620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f168c006010 tx=0x7f168c004040 comp rx=0 tx=0).stop 2024-04-21T10:28:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.780+0000 7f169ebe6640 1 -- 172.21.15.26:0/2681015871 shutdown_connections 2024-04-21T10:28:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.780+0000 7f169ebe6640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f16800a3bb0 0x7f16800a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.780+0000 7f169ebe6640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1698103b50 0x7f169819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.780+0000 7f169ebe6640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1698103190 0x7f1698198620 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.780+0000 7f169ebe6640 1 --2- 172.21.15.26:0/2681015871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1698101fa0 0x7f16981980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.780+0000 7f169ebe6640 1 -- 172.21.15.26:0/2681015871 >> 172.21.15.26:0/2681015871 conn(0x7f16980fd530 msgr2=0x7f1698106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.780+0000 7f169ebe6640 1 -- 172.21.15.26:0/2681015871 shutdown_connections 2024-04-21T10:28:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:43.780+0000 7f169ebe6640 1 -- 172.21.15.26:0/2681015871 wait complete. 2024-04-21T10:28:43.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:43 smithi026 ceph-mon[27965]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:43.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:43.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:43.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:43.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:43.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:43.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:43.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:43 smithi026 ceph-mon[27965]: Deploying daemon mds.h on smithi169 2024-04-21T10:28:43.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:43 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] up:boot 2024-04-21T10:28:43.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:43 smithi026 ceph-mon[27965]: fsmap 7 up:standby 2024-04-21T10:28:43.790 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:43 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T10:28:43.940 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:43 smithi169 ceph-mon[31469]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:43.940 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:43.941 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:43.941 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:43.941 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:43.941 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:43.941 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:43.941 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:43 smithi169 ceph-mon[31469]: Deploying daemon mds.h on smithi169 2024-04-21T10:28:43.942 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:43 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] up:boot 2024-04-21T10:28:43.942 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:43 smithi169 ceph-mon[31469]: fsmap 7 up:standby 2024-04-21T10:28:43.942 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:43 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T10:28:43.959 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:43 smithi190 ceph-mon[31423]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:43.960 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:43.960 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:43.960 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:43.960 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:43.960 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:43.960 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:43.960 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:43 smithi190 ceph-mon[31423]: Deploying daemon mds.h on smithi169 2024-04-21T10:28:43.960 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:43 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] up:boot 2024-04-21T10:28:43.960 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:43 smithi190 ceph-mon[31423]: fsmap 7 up:standby 2024-04-21T10:28:43.960 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:43 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T10:28:44.874 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:44 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2681015871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:44.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:44 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2681015871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:44.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:44 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2681015871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:44.948 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:28:44 smithi169 systemd[1]: Starting Ceph mds.h for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:28:45.271 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:28:45 smithi169 podman[61805]: 2024-04-21 10:28:44.924169439 +0000 UTC m=+0.018211245 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:28:45.271 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:28:45 smithi169 podman[61805]: 2024-04-21 10:28:45.027781265 +0000 UTC m=+0.121823062 container create 80036354afb85f5d05f7f4f304ee005c523a984d405ba35efe4a1bfa1273328b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T10:28:45.271 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:28:45 smithi169 podman[61805]: 2024-04-21 10:28:45.24480649 +0000 UTC m=+0.338848290 container init 80036354afb85f5d05f7f4f304ee005c523a984d405ba35efe4a1bfa1273328b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-21T10:28:45.271 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:28:45 smithi169 podman[61805]: 2024-04-21 10:28:45.247432428 +0000 UTC m=+0.341474228 container start 80036354afb85f5d05f7f4f304ee005c523a984d405ba35efe4a1bfa1273328b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:28:45.271 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:28:45 smithi169 ceph-mds[61824]: -- 172.21.15.169:0/3737989886 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561fe9a59a40 con 0x561fe9a2d000 2024-04-21T10:28:45.618 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:28:45 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-h[61820]: starting mds.h at 2024-04-21T10:28:45.618 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:28:45 smithi169 bash[61805]: 80036354afb85f5d05f7f4f304ee005c523a984d405ba35efe4a1bfa1273328b 2024-04-21T10:28:45.618 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:28:45 smithi169 systemd[1]: Started Ceph mds.h for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:28:45.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:45.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.907+0000 7f72fd8d2640 1 -- 172.21.15.26:0/726082510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72f8100610 msgr2=0x7f72f8100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:45.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.907+0000 7f72fd8d2640 1 --2- 172.21.15.26:0/726082510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72f8100610 0x7f72f8100a30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009900 tx=0x7f72e802bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:45.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.907+0000 7f72fd8d2640 1 -- 172.21.15.26:0/726082510 shutdown_connections 2024-04-21T10:28:45.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.907+0000 7f72fd8d2640 1 --2- 172.21.15.26:0/726082510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72f81021c0 0x7f72f8108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:45.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.907+0000 7f72fd8d2640 1 --2- 172.21.15.26:0/726082510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72f8101800 0x7f72f8101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:45.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.907+0000 7f72fd8d2640 1 --2- 172.21.15.26:0/726082510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72f8100610 0x7f72f8100a30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:45.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.907+0000 7f72fd8d2640 1 -- 172.21.15.26:0/726082510 >> 172.21.15.26:0/726082510 conn(0x7f72f80fbb80 msgr2=0x7f72f80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:45.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.907+0000 7f72fd8d2640 1 -- 172.21.15.26:0/726082510 shutdown_connections 2024-04-21T10:28:45.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.907+0000 7f72fd8d2640 1 -- 172.21.15.26:0/726082510 wait complete. 2024-04-21T10:28:45.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72fd8d2640 1 Processor -- start 2024-04-21T10:28:45.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72fd8d2640 1 -- start start 2024-04-21T10:28:45.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72fd8d2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72f8100610 0x7f72f81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:45.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72fd8d2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72f8101800 0x7f72f81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:45.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72fd8d2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72f81021c0 0x7f72f819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:45.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72fd8d2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f72f810c190 con 0x7f72f81021c0 2024-04-21T10:28:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72fd8d2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72f810c010 con 0x7f72f8100610 2024-04-21T10:28:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72fd8d2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f72f810c310 con 0x7f72f8101800 2024-04-21T10:28:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72f81021c0 0x7f72f819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72f81021c0 0x7f72f819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33232/0 (socket says 172.21.15.26:33232) 2024-04-21T10:28:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72f77fe640 1 -- 172.21.15.26:0/1234048491 learned_addr learned my addr 172.21.15.26:0/1234048491 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:45.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72f6ffd640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72f8100610 0x7f72f81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:45.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.908+0000 7f72f67fc640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72f8101800 0x7f72f81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:45.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.909+0000 7f72f77fe640 1 -- 172.21.15.26:0/1234048491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72f8101800 msgr2=0x7f72f81986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:45.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.909+0000 7f72f77fe640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72f8101800 0x7f72f81986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:45.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.909+0000 7f72f77fe640 1 -- 172.21.15.26:0/1234048491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72f8100610 msgr2=0x7f72f81981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:45.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.909+0000 7f72f77fe640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72f8100610 0x7f72f81981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:45.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.909+0000 7f72f77fe640 1 -- 172.21.15.26:0/1234048491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f819fdd0 con 0x7f72f81021c0 2024-04-21T10:28:45.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.909+0000 7f72f6ffd640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72f8100610 0x7f72f81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:28:45.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.909+0000 7f72f77fe640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72f81021c0 0x7f72f819f760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f72f000eaa0 tx=0x7f72f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:45.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.909+0000 7f72fc8d0640 1 -- 172.21.15.26:0/1234048491 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f000cdb0 con 0x7f72f81021c0 2024-04-21T10:28:45.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.909+0000 7f72fc8d0640 1 -- 172.21.15.26:0/1234048491 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72f0005020 con 0x7f72f81021c0 2024-04-21T10:28:45.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.909+0000 7f72fd8d2640 1 -- 172.21.15.26:0/1234048491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f81a00c0 con 0x7f72f81021c0 2024-04-21T10:28:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.909+0000 7f72fc8d0640 1 -- 172.21.15.26:0/1234048491 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f00116d0 con 0x7f72f81021c0 2024-04-21T10:28:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.909+0000 7f72fd8d2640 1 -- 172.21.15.26:0/1234048491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f81a0610 con 0x7f72f81021c0 2024-04-21T10:28:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.910+0000 7f72fd8d2640 1 -- 172.21.15.26:0/1234048491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72f8108350 con 0x7f72f81021c0 2024-04-21T10:28:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.911+0000 7f72fc8d0640 1 -- 172.21.15.26:0/1234048491 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f72f00051d0 con 0x7f72f81021c0 2024-04-21T10:28:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.912+0000 7f72fc8d0640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f72e00a3bb0 0x7f72e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.912+0000 7f72f6ffd640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f72e00a3bb0 0x7f72e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.913+0000 7f72fc8d0640 1 -- 172.21.15.26:0/1234048491 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f72f00d0070 con 0x7f72f81021c0 2024-04-21T10:28:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.913+0000 7f72f6ffd640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f72e00a3bb0 0x7f72e00a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f72e802c240 tx=0x7f72e802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:45.917+0000 7f72fc8d0640 1 -- 172.21.15.26:0/1234048491 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72f0016030 con 0x7f72f81021c0 2024-04-21T10:28:45.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:45 smithi026 ceph-mon[27965]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:45.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:45 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:45.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:45 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:45.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:45 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:45.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:45 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:45.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:45 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:45.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:45 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:45.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:45 smithi169 ceph-mon[31469]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:45.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:45 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:45.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:45 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:45.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:45 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:45.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:45 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:45.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:45 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:45.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:45 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:46.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:45 smithi190 ceph-mon[31423]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:46.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:45 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:46.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:45 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:46.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:45 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:46.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:45 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:46.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:45 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:46.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:45 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:46.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.143+0000 7f72fd8d2640 1 -- 172.21.15.26:0/1234048491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72f8062b50 con 0x7f72f81021c0 2024-04-21T10:28:46.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.143+0000 7f72fc8d0640 1 -- 172.21.15.26:0/1234048491 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7f72f009c8a0 con 0x7f72f81021c0 2024-04-21T10:28:46.143 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:46.143 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-21T10:28:46.144 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 9 2024-04-21T10:28:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.145+0000 7f72fd8d2640 1 -- 172.21.15.26:0/1234048491 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f72e00a3bb0 msgr2=0x7f72e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.145+0000 7f72fd8d2640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f72e00a3bb0 0x7f72e00a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f72e802c240 tx=0x7f72e802c7e0 comp rx=0 tx=0).stop 2024-04-21T10:28:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.145+0000 7f72fd8d2640 1 -- 172.21.15.26:0/1234048491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72f81021c0 msgr2=0x7f72f819f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.145+0000 7f72fd8d2640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72f81021c0 0x7f72f819f760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f72f000eaa0 tx=0x7f72f000ef60 comp rx=0 tx=0).stop 2024-04-21T10:28:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.145+0000 7f72fd8d2640 1 -- 172.21.15.26:0/1234048491 shutdown_connections 2024-04-21T10:28:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.145+0000 7f72fd8d2640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f72e00a3bb0 0x7f72e00a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.145+0000 7f72fd8d2640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72f81021c0 0x7f72f819f760 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.145+0000 7f72fd8d2640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72f8101800 0x7f72f81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.145+0000 7f72fd8d2640 1 --2- 172.21.15.26:0/1234048491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72f8100610 0x7f72f81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.145+0000 7f72fd8d2640 1 -- 172.21.15.26:0/1234048491 >> 172.21.15.26:0/1234048491 conn(0x7f72f80fbb80 msgr2=0x7f72f810ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:46.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.146+0000 7f72fd8d2640 1 -- 172.21.15.26:0/1234048491 shutdown_connections 2024-04-21T10:28:46.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:46.146+0000 7f72fd8d2640 1 -- 172.21.15.26:0/1234048491 wait complete. 2024-04-21T10:28:46.648 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:46 smithi169 ceph-mon[31469]: Deploying daemon mds.i on smithi190 2024-04-21T10:28:46.648 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:46 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.169:6836/2325983283,v1:172.21.15.169:6837/2325983283] up:boot 2024-04-21T10:28:46.648 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:46 smithi169 ceph-mon[31469]: fsmap 8 up:standby 2024-04-21T10:28:46.648 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:46 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T10:28:46.648 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:46 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1234048491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:46.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:46 smithi026 ceph-mon[27965]: Deploying daemon mds.i on smithi190 2024-04-21T10:28:46.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:46 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.169:6836/2325983283,v1:172.21.15.169:6837/2325983283] up:boot 2024-04-21T10:28:46.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:46 smithi026 ceph-mon[27965]: fsmap 8 up:standby 2024-04-21T10:28:46.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:46 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T10:28:46.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:46 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1234048491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:47.021 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:46 smithi190 ceph-mon[31423]: Deploying daemon mds.i on smithi190 2024-04-21T10:28:47.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:46 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.169:6836/2325983283,v1:172.21.15.169:6837/2325983283] up:boot 2024-04-21T10:28:47.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:46 smithi190 ceph-mon[31423]: fsmap 8 up:standby 2024-04-21T10:28:47.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:46 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T10:28:47.022 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:46 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1234048491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:47.624 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:28:47 smithi190 systemd[1]: Starting Ceph mds.i for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:28:47.935 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:47 smithi190 ceph-mon[31423]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:47.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:47 smithi026 ceph-mon[27965]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:47.935 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:28:47 smithi190 podman[62734]: 2024-04-21 10:28:47.610104621 +0000 UTC m=+0.018229645 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:28:47.936 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:28:47 smithi190 podman[62734]: 2024-04-21 10:28:47.717232039 +0000 UTC m=+0.125357054 container create fc172937a855998c9b00e3851472e828b6e8a35fa7314ecb3e9ec9768f637bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-21T10:28:47.936 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:28:47 smithi190 podman[62734]: 2024-04-21 10:28:47.936413164 +0000 UTC m=+0.344538176 container init fc172937a855998c9b00e3851472e828b6e8a35fa7314ecb3e9ec9768f637bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-21T10:28:47.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:47 smithi169 ceph-mon[31469]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:48.157 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.266+0000 7f247ee11640 1 -- 172.21.15.26:0/867546685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478102980 msgr2=0x7f24781092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.266+0000 7f247ee11640 1 --2- 172.21.15.26:0/867546685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478102980 0x7f24781092a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2470009ca0 tx=0x7f247002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:28:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.266+0000 7f247ee11640 1 -- 172.21.15.26:0/867546685 shutdown_connections 2024-04-21T10:28:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.266+0000 7f247ee11640 1 --2- 172.21.15.26:0/867546685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478102980 0x7f24781092a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.266+0000 7f247ee11640 1 --2- 172.21.15.26:0/867546685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2478101fc0 0x7f2478102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.266+0000 7f247ee11640 1 --2- 172.21.15.26:0/867546685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2478100e70 0x7f2478101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.266+0000 7f247ee11640 1 -- 172.21.15.26:0/867546685 >> 172.21.15.26:0/867546685 conn(0x7f24780fc3e0 msgr2=0x7f24780fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.266+0000 7f247ee11640 1 -- 172.21.15.26:0/867546685 shutdown_connections 2024-04-21T10:28:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.266+0000 7f247ee11640 1 -- 172.21.15.26:0/867546685 wait complete. 2024-04-21T10:28:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.266+0000 7f247ee11640 1 Processor -- start 2024-04-21T10:28:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247ee11640 1 -- start start 2024-04-21T10:28:48.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247ee11640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478100e70 0x7f247819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247ee11640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2478101fc0 0x7f247819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247ee11640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2478102980 0x7f24781a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247ee11640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f247810c9c0 con 0x7f2478100e70 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247ee11640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f247810c840 con 0x7f2478102980 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247ee11640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f247810cb40 con 0x7f2478101fc0 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247cbad640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478100e70 0x7f247819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247cbad640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478100e70 0x7f247819a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33250/0 (socket says 172.21.15.26:33250) 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247cbad640 1 -- 172.21.15.26:0/2516511606 learned_addr learned my addr 172.21.15.26:0/2516511606 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247d3ae640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2478102980 0x7f24781a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f2477fff640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2478101fc0 0x7f247819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247cbad640 1 -- 172.21.15.26:0/2516511606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2478101fc0 msgr2=0x7f247819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247cbad640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2478101fc0 0x7f247819a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247cbad640 1 -- 172.21.15.26:0/2516511606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2478102980 msgr2=0x7f24781a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247cbad640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2478102980 0x7f24781a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:48.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247cbad640 1 -- 172.21.15.26:0/2516511606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24781a1f80 con 0x7f2478100e70 2024-04-21T10:28:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f247cbad640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478100e70 0x7f247819a360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f246c00e9f0 tx=0x7f246c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.267+0000 7f2475ffb640 1 -- 172.21.15.26:0/2516511606 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c00cd80 con 0x7f2478100e70 2024-04-21T10:28:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.268+0000 7f2475ffb640 1 -- 172.21.15.26:0/2516511606 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f246c0054d0 con 0x7f2478100e70 2024-04-21T10:28:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.268+0000 7f247ee11640 1 -- 172.21.15.26:0/2516511606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24781a2270 con 0x7f2478100e70 2024-04-21T10:28:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.268+0000 7f2475ffb640 1 -- 172.21.15.26:0/2516511606 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c010690 con 0x7f2478100e70 2024-04-21T10:28:48.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.268+0000 7f247ee11640 1 -- 172.21.15.26:0/2516511606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24781a27c0 con 0x7f2478100e70 2024-04-21T10:28:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.269+0000 7f247ee11640 1 -- 172.21.15.26:0/2516511606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24781a2a80 con 0x7f2478100e70 2024-04-21T10:28:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.270+0000 7f2475ffb640 1 -- 172.21.15.26:0/2516511606 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f246c004e90 con 0x7f2478100e70 2024-04-21T10:28:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.271+0000 7f2475ffb640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f24680a3c30 0x7f24680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.271+0000 7f2477fff640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f24680a3c30 0x7f24680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.271+0000 7f2475ffb640 1 -- 172.21.15.26:0/2516511606 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f246c0d0410 con 0x7f2478100e70 2024-04-21T10:28:48.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.271+0000 7f2477fff640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f24680a3c30 0x7f24680a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2464005fd0 tx=0x7f2464005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:48.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.275+0000 7f2475ffb640 1 -- 172.21.15.26:0/2516511606 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f246c09d290 con 0x7f2478100e70 2024-04-21T10:28:48.276 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:28:47 smithi190 podman[62734]: 2024-04-21 10:28:47.938939008 +0000 UTC m=+0.347064030 container start fc172937a855998c9b00e3851472e828b6e8a35fa7314ecb3e9ec9768f637bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-21T10:28:48.276 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:28:47 smithi190 ceph-mds[62753]: -- 172.21.15.190:0/3587210451 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559b054dba40 con 0x559b054af000 2024-04-21T10:28:48.276 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:28:47 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i[62749]: starting mds.i at 2024-04-21T10:28:48.277 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:28:48 smithi190 bash[62734]: fc172937a855998c9b00e3851472e828b6e8a35fa7314ecb3e9ec9768f637bd1 2024-04-21T10:28:48.277 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:28:48 smithi190 systemd[1]: Started Ceph mds.i for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:28:48.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.519+0000 7f247ee11640 1 -- 172.21.15.26:0/2516511606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24781a2c50 con 0x7f2478100e70 2024-04-21T10:28:48.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.520+0000 7f2475ffb640 1 -- 172.21.15.26:0/2516511606 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7f246c09cb90 con 0x7f2478100e70 2024-04-21T10:28:48.522 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:48.522 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-21T10:28:48.523 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 9 2024-04-21T10:28:48.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.522+0000 7f247ee11640 1 -- 172.21.15.26:0/2516511606 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f24680a3c30 msgr2=0x7f24680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:48.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.522+0000 7f247ee11640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f24680a3c30 0x7f24680a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2464005fd0 tx=0x7f2464005850 comp rx=0 tx=0).stop 2024-04-21T10:28:48.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.522+0000 7f247ee11640 1 -- 172.21.15.26:0/2516511606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478100e70 msgr2=0x7f247819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:48.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.522+0000 7f247ee11640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478100e70 0x7f247819a360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f246c00e9f0 tx=0x7f246c00edf0 comp rx=0 tx=0).stop 2024-04-21T10:28:48.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.523+0000 7f247ee11640 1 -- 172.21.15.26:0/2516511606 shutdown_connections 2024-04-21T10:28:48.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.523+0000 7f247ee11640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f24680a3c30 0x7f24680a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:48.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.523+0000 7f247ee11640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2478102980 0x7f24781a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:48.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.523+0000 7f247ee11640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2478101fc0 0x7f247819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:48.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.523+0000 7f247ee11640 1 --2- 172.21.15.26:0/2516511606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478100e70 0x7f247819a360 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:48.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.523+0000 7f247ee11640 1 -- 172.21.15.26:0/2516511606 >> 172.21.15.26:0/2516511606 conn(0x7f24780fc3e0 msgr2=0x7f24780fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:48.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.523+0000 7f247ee11640 1 -- 172.21.15.26:0/2516511606 shutdown_connections 2024-04-21T10:28:48.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:48.523+0000 7f247ee11640 1 -- 172.21.15.26:0/2516511606 wait complete. 2024-04-21T10:28:49.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:49.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:49.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:49.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:49.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:49.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:49.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:49 smithi026 ceph-mon[27965]: Deploying daemon mds.g on smithi026 2024-04-21T10:28:49.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:49 smithi026 ceph-mon[27965]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:49.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:49 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2516511606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:49.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:49 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] up:boot 2024-04-21T10:28:49.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:49 smithi026 ceph-mon[27965]: fsmap 9 up:standby 2024-04-21T10:28:49.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:49 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T10:28:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:49 smithi169 ceph-mon[31469]: Deploying daemon mds.g on smithi026 2024-04-21T10:28:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:49 smithi169 ceph-mon[31469]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:49.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:49 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2516511606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:49.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:49 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] up:boot 2024-04-21T10:28:49.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:49 smithi169 ceph-mon[31469]: fsmap 9 up:standby 2024-04-21T10:28:49.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:49 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:49 smithi190 ceph-mon[31423]: Deploying daemon mds.g on smithi026 2024-04-21T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:49 smithi190 ceph-mon[31423]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:49 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2516511606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:49 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] up:boot 2024-04-21T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:49 smithi190 ceph-mon[31423]: fsmap 9 up:standby 2024-04-21T10:28:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:49 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T10:28:50.435 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:28:50 smithi026 systemd[1]: Starting Ceph mds.g for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:28:50.535 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:50.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.642+0000 7f5527b6d640 1 -- 172.21.15.26:0/1806122704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520100ea0 msgr2=0x7f55201012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:50.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.642+0000 7f5527b6d640 1 --2- 172.21.15.26:0/1806122704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520100ea0 0x7f55201012c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5510009900 tx=0x7f551002bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:50.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.642+0000 7f5527b6d640 1 -- 172.21.15.26:0/1806122704 shutdown_connections 2024-04-21T10:28:50.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.642+0000 7f5527b6d640 1 --2- 172.21.15.26:0/1806122704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55201029b0 0x7f55201092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:50.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.642+0000 7f5527b6d640 1 --2- 172.21.15.26:0/1806122704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5520101ff0 0x7f5520102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:50.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.642+0000 7f5527b6d640 1 --2- 172.21.15.26:0/1806122704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520100ea0 0x7f55201012c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:50.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.642+0000 7f5527b6d640 1 -- 172.21.15.26:0/1806122704 >> 172.21.15.26:0/1806122704 conn(0x7f55200fc410 msgr2=0x7f55200fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:50.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.642+0000 7f5527b6d640 1 -- 172.21.15.26:0/1806122704 shutdown_connections 2024-04-21T10:28:50.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.642+0000 7f5527b6d640 1 -- 172.21.15.26:0/1806122704 wait complete. 2024-04-21T10:28:50.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.642+0000 7f5527b6d640 1 Processor -- start 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.642+0000 7f5527b6d640 1 -- start start 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5527b6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520100ea0 0x7f5520197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5527b6d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5520101ff0 0x7f5520197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5527b6d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55201029b0 0x7f552019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5527b6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f552010ca50 con 0x7f5520100ea0 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5527b6d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f552010c8d0 con 0x7f5520101ff0 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5527b6d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f552010cbd0 con 0x7f55201029b0 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5525909640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520100ea0 0x7f5520197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5525909640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520100ea0 0x7f5520197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33266/0 (socket says 172.21.15.26:33266) 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5525909640 1 -- 172.21.15.26:0/3105193658 learned_addr learned my addr 172.21.15.26:0/3105193658 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5525108640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5520101ff0 0x7f5520197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f552610a640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55201029b0 0x7f552019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5525909640 1 -- 172.21.15.26:0/3105193658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55201029b0 msgr2=0x7f552019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5525909640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55201029b0 0x7f552019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5525909640 1 -- 172.21.15.26:0/3105193658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5520101ff0 msgr2=0x7f5520197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5525909640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5520101ff0 0x7f5520197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5525909640 1 -- 172.21.15.26:0/3105193658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552019ec60 con 0x7f5520100ea0 2024-04-21T10:28:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5525108640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5520101ff0 0x7f5520197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:28:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5525909640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520100ea0 0x7f5520197040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5510036040 tx=0x7f55100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.643+0000 7f5516ffd640 1 -- 172.21.15.26:0/3105193658 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55100044e0 con 0x7f5520100ea0 2024-04-21T10:28:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.644+0000 7f5527b6d640 1 -- 172.21.15.26:0/3105193658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552006a640 con 0x7f5520100ea0 2024-04-21T10:28:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.644+0000 7f5516ffd640 1 -- 172.21.15.26:0/3105193658 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55100035f0 con 0x7f5520100ea0 2024-04-21T10:28:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.644+0000 7f5516ffd640 1 -- 172.21.15.26:0/3105193658 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55100476e0 con 0x7f5520100ea0 2024-04-21T10:28:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.644+0000 7f5527b6d640 1 -- 172.21.15.26:0/3105193658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552006ac10 con 0x7f5520100ea0 2024-04-21T10:28:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.645+0000 7f5527b6d640 1 -- 172.21.15.26:0/3105193658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5520107bc0 con 0x7f5520100ea0 2024-04-21T10:28:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.646+0000 7f5516ffd640 1 -- 172.21.15.26:0/3105193658 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5510003a50 con 0x7f5520100ea0 2024-04-21T10:28:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.647+0000 7f5516ffd640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f550c0a3bb0 0x7f550c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.647+0000 7f5516ffd640 1 -- 172.21.15.26:0/3105193658 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f55100f4840 con 0x7f5520100ea0 2024-04-21T10:28:50.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.647+0000 7f5525108640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f550c0a3bb0 0x7f550c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:50.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.647+0000 7f5525108640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f550c0a3bb0 0x7f550c0a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5520198550 tx=0x7f551c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:50.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.651+0000 7f5516ffd640 1 -- 172.21.15.26:0/3105193658 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55100c16c0 con 0x7f5520100ea0 2024-04-21T10:28:50.795 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:28:50 smithi026 podman[66597]: 2024-04-21 10:28:50.423767886 +0000 UTC m=+0.018651554 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:28:50.795 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:28:50 smithi026 podman[66597]: 2024-04-21 10:28:50.544046709 +0000 UTC m=+0.138930366 container create 9494afbdcc141d2846ab7b680dd46c5fc42de262f43cdc14701e7d4b4579d74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-21T10:28:50.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.885+0000 7f5527b6d640 1 -- 172.21.15.26:0/3105193658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5520107de0 con 0x7f5520100ea0 2024-04-21T10:28:50.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.886+0000 7f5516ffd640 1 -- 172.21.15.26:0/3105193658 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7f55100c0fc0 con 0x7f5520100ea0 2024-04-21T10:28:50.886 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:50.887 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T10:28:50.887 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 10 2024-04-21T10:28:50.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.888+0000 7f5527b6d640 1 -- 172.21.15.26:0/3105193658 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f550c0a3bb0 msgr2=0x7f550c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:50.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.888+0000 7f5527b6d640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f550c0a3bb0 0x7f550c0a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5520198550 tx=0x7f551c005db0 comp rx=0 tx=0).stop 2024-04-21T10:28:50.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.888+0000 7f5527b6d640 1 -- 172.21.15.26:0/3105193658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520100ea0 msgr2=0x7f5520197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:50.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.888+0000 7f5527b6d640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520100ea0 0x7f5520197040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5510036040 tx=0x7f55100042f0 comp rx=0 tx=0).stop 2024-04-21T10:28:50.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.888+0000 7f5527b6d640 1 -- 172.21.15.26:0/3105193658 shutdown_connections 2024-04-21T10:28:50.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.888+0000 7f5527b6d640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f550c0a3bb0 0x7f550c0a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:50.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.888+0000 7f5527b6d640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55201029b0 0x7f552019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:50.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.888+0000 7f5527b6d640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5520101ff0 0x7f5520197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:50.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.888+0000 7f5527b6d640 1 --2- 172.21.15.26:0/3105193658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520100ea0 0x7f5520197040 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:50.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.888+0000 7f5527b6d640 1 -- 172.21.15.26:0/3105193658 >> 172.21.15.26:0/3105193658 conn(0x7f55200fc410 msgr2=0x7f5520105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:50.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.888+0000 7f5527b6d640 1 -- 172.21.15.26:0/3105193658 shutdown_connections 2024-04-21T10:28:50.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:50.888+0000 7f5527b6d640 1 -- 172.21.15.26:0/3105193658 wait complete. 2024-04-21T10:28:51.185 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:28:50 smithi026 podman[66597]: 2024-04-21 10:28:50.796905212 +0000 UTC m=+0.391788871 container init 9494afbdcc141d2846ab7b680dd46c5fc42de262f43cdc14701e7d4b4579d74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-21T10:28:51.185 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:28:50 smithi026 podman[66597]: 2024-04-21 10:28:50.80018201 +0000 UTC m=+0.395065669 container start 9494afbdcc141d2846ab7b680dd46c5fc42de262f43cdc14701e7d4b4579d74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-04-21T10:28:51.185 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:28:50 smithi026 ceph-mds[66654]: -- 172.21.15.26:0/2372604781 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e73843a40 con 0x564e73816800 2024-04-21T10:28:51.185 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:28:50 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-g[66650]: starting mds.g at 2024-04-21T10:28:51.185 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:28:50 smithi026 bash[66597]: 9494afbdcc141d2846ab7b680dd46c5fc42de262f43cdc14701e7d4b4579d74c 2024-04-21T10:28:51.185 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:28:50 smithi026 systemd[1]: Started Ceph mds.g for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:28:51.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:51 smithi190 ceph-mon[31423]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:51.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:51 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3105193658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:51.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:51.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:51.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:51.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:51.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:51.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:51 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:51.630 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:51 smithi026 ceph-mon[27965]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:51.630 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:51 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3105193658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:51.630 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:51.630 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:51.630 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:51.630 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:51.630 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:51.630 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:51 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:51.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:51 smithi169 ceph-mon[31469]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:51.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:51 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3105193658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:51.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:51.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:51.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:51.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:51.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:51.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:51 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:52.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:52 smithi026 ceph-mon[27965]: Deploying daemon mds.a on smithi026 2024-04-21T10:28:52.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:52 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.26:6838/3108850523,v1:172.21.15.26:6839/3108850523] up:boot 2024-04-21T10:28:52.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:52 smithi026 ceph-mon[27965]: fsmap 10 up:standby 2024-04-21T10:28:52.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:52 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T10:28:52.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:52 smithi190 ceph-mon[31423]: Deploying daemon mds.a on smithi026 2024-04-21T10:28:52.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:52 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.26:6838/3108850523,v1:172.21.15.26:6839/3108850523] up:boot 2024-04-21T10:28:52.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:52 smithi190 ceph-mon[31423]: fsmap 10 up:standby 2024-04-21T10:28:52.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:52 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T10:28:52.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:52 smithi169 ceph-mon[31469]: Deploying daemon mds.a on smithi026 2024-04-21T10:28:52.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:52 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.26:6838/3108850523,v1:172.21.15.26:6839/3108850523] up:boot 2024-04-21T10:28:52.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:52 smithi169 ceph-mon[31469]: fsmap 10 up:standby 2024-04-21T10:28:52.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:52 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T10:28:52.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:53.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.118+0000 7f0bdbed2640 1 -- 172.21.15.26:0/24125577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bd4101fc0 msgr2=0x7f0bd4102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:53.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.118+0000 7f0bdbed2640 1 --2- 172.21.15.26:0/24125577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bd4101fc0 0x7f0bd4102440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009900 tx=0x7f0bc402bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:53.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.118+0000 7f0bdbed2640 1 -- 172.21.15.26:0/24125577 shutdown_connections 2024-04-21T10:28:53.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.118+0000 7f0bdbed2640 1 --2- 172.21.15.26:0/24125577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd4102980 0x7f0bd41092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:53.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.118+0000 7f0bdbed2640 1 --2- 172.21.15.26:0/24125577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bd4101fc0 0x7f0bd4102440 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:53.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.118+0000 7f0bdbed2640 1 --2- 172.21.15.26:0/24125577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bd4100e70 0x7f0bd4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:53.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.118+0000 7f0bdbed2640 1 -- 172.21.15.26:0/24125577 >> 172.21.15.26:0/24125577 conn(0x7f0bd40fc3e0 msgr2=0x7f0bd40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:53.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.118+0000 7f0bdbed2640 1 -- 172.21.15.26:0/24125577 shutdown_connections 2024-04-21T10:28:53.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.118+0000 7f0bdbed2640 1 -- 172.21.15.26:0/24125577 wait complete. 2024-04-21T10:28:53.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.118+0000 7f0bdbed2640 1 Processor -- start 2024-04-21T10:28:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.118+0000 7f0bdbed2640 1 -- start start 2024-04-21T10:28:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bdbed2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd4100e70 0x7f0bd4069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bdbed2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bd4101fc0 0x7f0bd419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bdbed2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bd4102980 0x7f0bd419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bdbed2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd410c9c0 con 0x7f0bd4101fc0 2024-04-21T10:28:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bdbed2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd410c840 con 0x7f0bd4100e70 2024-04-21T10:28:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bdbed2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd410cb40 con 0x7f0bd4102980 2024-04-21T10:28:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bd946d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bd4101fc0 0x7f0bd419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bd946d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bd4101fc0 0x7f0bd419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33308/0 (socket says 172.21.15.26:33308) 2024-04-21T10:28:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bd946d640 1 -- 172.21.15.26:0/3611191748 learned_addr learned my addr 172.21.15.26:0/3611191748 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bda46f640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bd4102980 0x7f0bd419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:53.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bd9c6e640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd4100e70 0x7f0bd4069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bd946d640 1 -- 172.21.15.26:0/3611191748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bd4102980 msgr2=0x7f0bd419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bd946d640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bd4102980 0x7f0bd419f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bd946d640 1 -- 172.21.15.26:0/3611191748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd4100e70 msgr2=0x7f0bd4069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bd946d640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd4100e70 0x7f0bd4069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bd946d640 1 -- 172.21.15.26:0/3611191748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd419fde0 con 0x7f0bd4101fc0 2024-04-21T10:28:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.119+0000 7f0bd946d640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bd4101fc0 0x7f0bd419ceb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4000c00 tx=0x7f0bc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.120+0000 7f0bcaffd640 1 -- 172.21.15.26:0/3611191748 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4043020 con 0x7f0bd4101fc0 2024-04-21T10:28:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.120+0000 7f0bdbed2640 1 -- 172.21.15.26:0/3611191748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd41a0010 con 0x7f0bd4101fc0 2024-04-21T10:28:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.120+0000 7f0bcaffd640 1 -- 172.21.15.26:0/3611191748 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bc4004aa0 con 0x7f0bd4101fc0 2024-04-21T10:28:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.120+0000 7f0bcaffd640 1 -- 172.21.15.26:0/3611191748 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4003d00 con 0x7f0bd4101fc0 2024-04-21T10:28:53.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.120+0000 7f0bdbed2640 1 -- 172.21.15.26:0/3611191748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd41a0560 con 0x7f0bd4101fc0 2024-04-21T10:28:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.121+0000 7f0bdbed2640 1 -- 172.21.15.26:0/3611191748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd4108200 con 0x7f0bd4101fc0 2024-04-21T10:28:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.122+0000 7f0bcaffd640 1 -- 172.21.15.26:0/3611191748 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0bc4046020 con 0x7f0bd4101fc0 2024-04-21T10:28:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.123+0000 7f0bcaffd640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0bc00a3bb0 0x7f0bc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.123+0000 7f0bcaffd640 1 -- 172.21.15.26:0/3611191748 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f0bc40f29a0 con 0x7f0bd4101fc0 2024-04-21T10:28:53.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.123+0000 7f0bd9c6e640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0bc00a3bb0 0x7f0bc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:53.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.124+0000 7f0bd9c6e640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0bc00a3bb0 0x7f0bc00a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005fd0 tx=0x7f0bd0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:53.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.127+0000 7f0bcaffd640 1 -- 172.21.15.26:0/3611191748 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0bc404d020 con 0x7f0bd4101fc0 2024-04-21T10:28:53.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.363+0000 7f0bdbed2640 1 -- 172.21.15.26:0/3611191748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bd402d5c0 con 0x7f0bd4101fc0 2024-04-21T10:28:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.363+0000 7f0bcaffd640 1 -- 172.21.15.26:0/3611191748 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7f0bc40bf800 con 0x7f0bd4101fc0 2024-04-21T10:28:53.363 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:53.363 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T10:28:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 11 2024-04-21T10:28:53.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.365+0000 7f0bdbed2640 1 -- 172.21.15.26:0/3611191748 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0bc00a3bb0 msgr2=0x7f0bc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:53.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.365+0000 7f0bdbed2640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0bc00a3bb0 0x7f0bc00a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0005fd0 tx=0x7f0bd0005850 comp rx=0 tx=0).stop 2024-04-21T10:28:53.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.365+0000 7f0bdbed2640 1 -- 172.21.15.26:0/3611191748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bd4101fc0 msgr2=0x7f0bd419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.365+0000 7f0bdbed2640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bd4101fc0 0x7f0bd419ceb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4000c00 tx=0x7f0bc4004270 comp rx=0 tx=0).stop 2024-04-21T10:28:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.365+0000 7f0bdbed2640 1 -- 172.21.15.26:0/3611191748 shutdown_connections 2024-04-21T10:28:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.365+0000 7f0bdbed2640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0bc00a3bb0 0x7f0bc00a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.365+0000 7f0bdbed2640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bd4102980 0x7f0bd419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.365+0000 7f0bdbed2640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bd4101fc0 0x7f0bd419ceb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.365+0000 7f0bdbed2640 1 --2- 172.21.15.26:0/3611191748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd4100e70 0x7f0bd4069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.365+0000 7f0bdbed2640 1 -- 172.21.15.26:0/3611191748 >> 172.21.15.26:0/3611191748 conn(0x7f0bd40fc3e0 msgr2=0x7f0bd41053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.366+0000 7f0bdbed2640 1 -- 172.21.15.26:0/3611191748 shutdown_connections 2024-04-21T10:28:53.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:53.366+0000 7f0bdbed2640 1 -- 172.21.15.26:0/3611191748 wait complete. 2024-04-21T10:28:53.375 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:53 smithi026 ceph-mon[27965]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:53.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:53 smithi190 ceph-mon[31423]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:53.685 INFO:journalctl@ceph.mds.a.smithi026.stdout:Apr 21 10:28:53 smithi026 systemd[1]: Started Ceph mds.a for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:28:53.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:53 smithi169 ceph-mon[31469]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:54.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:54 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3611191748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:54.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:54 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:54.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:54 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:54.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:54 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:54.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:54 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:54.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:54 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:54.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:54 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:54.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:54 smithi026 ceph-mon[27965]: Deploying daemon mds.b on smithi169 2024-04-21T10:28:54.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:54 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3611191748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:54.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:54 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:54.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:54 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:54.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:54 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:54.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:54 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:54.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:54 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:54.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:54 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:54.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:54 smithi190 ceph-mon[31423]: Deploying daemon mds.b on smithi169 2024-04-21T10:28:54.588 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:54 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3611191748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:54.588 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:54.588 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:54.588 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:54.589 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T10:28:54.589 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T10:28:54.589 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:54 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:54.589 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:54 smithi169 ceph-mon[31469]: Deploying daemon mds.b on smithi169 2024-04-21T10:28:55.378 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.496+0000 7fb144c6d640 1 -- 172.21.15.26:0/3478862906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140107ca0 msgr2=0x7fb14010a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.496+0000 7fb144c6d640 1 --2- 172.21.15.26:0/3478862906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140107ca0 0x7fb14010a0f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb138009ca0 tx=0x7fb13802c0f0 comp rx=0 tx=0).stop 2024-04-21T10:28:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.496+0000 7fb144c6d640 1 -- 172.21.15.26:0/3478862906 shutdown_connections 2024-04-21T10:28:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.496+0000 7fb144c6d640 1 --2- 172.21.15.26:0/3478862906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140107ca0 0x7fb14010a0f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.496+0000 7fb144c6d640 1 --2- 172.21.15.26:0/3478862906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb140105370 0x7fb140107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.496+0000 7fb144c6d640 1 --2- 172.21.15.26:0/3478862906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb140102a40 0x7fb140104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.496+0000 7fb144c6d640 1 -- 172.21.15.26:0/3478862906 >> 172.21.15.26:0/3478862906 conn(0x7fb1400fc470 msgr2=0x7fb1400fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.496+0000 7fb144c6d640 1 -- 172.21.15.26:0/3478862906 shutdown_connections 2024-04-21T10:28:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.496+0000 7fb144c6d640 1 -- 172.21.15.26:0/3478862906 wait complete. 2024-04-21T10:28:55.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.497+0000 7fb144c6d640 1 Processor -- start 2024-04-21T10:28:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.497+0000 7fb144c6d640 1 -- start start 2024-04-21T10:28:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.497+0000 7fb144c6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140102a40 0x7fb14019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.497+0000 7fb144c6d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb140105370 0x7fb14019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.497+0000 7fb144c6d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb140107ca0 0x7fb1401a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.497+0000 7fb144c6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb14010db00 con 0x7fb140102a40 2024-04-21T10:28:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.497+0000 7fb144c6d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb14010d980 con 0x7fb140105370 2024-04-21T10:28:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.497+0000 7fb144c6d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb14010dc80 con 0x7fb140107ca0 2024-04-21T10:28:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.497+0000 7fb13e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140102a40 0x7fb14019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb13e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140102a40 0x7fb14019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58516/0 (socket says 172.21.15.26:58516) 2024-04-21T10:28:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb13e59c640 1 -- 172.21.15.26:0/1489631306 learned_addr learned my addr 172.21.15.26:0/1489631306 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb137fff640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb140105370 0x7fb14019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb13ed9d640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb140107ca0 0x7fb1401a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:55.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb13e59c640 1 -- 172.21.15.26:0/1489631306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb140107ca0 msgr2=0x7fb1401a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:55.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb13e59c640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb140107ca0 0x7fb1401a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:55.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb13e59c640 1 -- 172.21.15.26:0/1489631306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb140105370 msgr2=0x7fb14019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:55.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb13e59c640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb140105370 0x7fb14019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:55.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb13e59c640 1 -- 172.21.15.26:0/1489631306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1401a1f70 con 0x7fb140102a40 2024-04-21T10:28:55.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb13ed9d640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb140107ca0 0x7fb1401a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:28:55.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb13e59c640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140102a40 0x7fb14019a350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb13000d770 tx=0x7fb13000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:55.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb1377fe640 1 -- 172.21.15.26:0/1489631306 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130017070 con 0x7fb140102a40 2024-04-21T10:28:55.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb1377fe640 1 -- 172.21.15.26:0/1489631306 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1300054d0 con 0x7fb140102a40 2024-04-21T10:28:55.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb1377fe640 1 -- 172.21.15.26:0/1489631306 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130004c30 con 0x7fb140102a40 2024-04-21T10:28:55.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb144c6d640 1 -- 172.21.15.26:0/1489631306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1401a2260 con 0x7fb140102a40 2024-04-21T10:28:55.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.498+0000 7fb144c6d640 1 -- 172.21.15.26:0/1489631306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1401a27b0 con 0x7fb140102a40 2024-04-21T10:28:55.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.499+0000 7fb144c6d640 1 -- 172.21.15.26:0/1489631306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb140109790 con 0x7fb140102a40 2024-04-21T10:28:55.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.501+0000 7fb1377fe640 1 -- 172.21.15.26:0/1489631306 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb130004dd0 con 0x7fb140102a40 2024-04-21T10:28:55.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.501+0000 7fb1377fe640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb12c0a3c30 0x7fb12c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:55.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.501+0000 7fb137fff640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb12c0a3c30 0x7fb12c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:55.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.501+0000 7fb1377fe640 1 -- 172.21.15.26:0/1489631306 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb1300cfdd0 con 0x7fb140102a40 2024-04-21T10:28:55.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.502+0000 7fb137fff640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb12c0a3c30 0x7fb12c0a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb14019b860 tx=0x7fb12800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:55.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.505+0000 7fb1377fe640 1 -- 172.21.15.26:0/1489631306 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb13009ccd0 con 0x7fb140102a40 2024-04-21T10:28:55.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:55 smithi190 ceph-mon[31423]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:55.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:55 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.26:6840/1075023600,v1:172.21.15.26:6841/1075023600] up:boot 2024-04-21T10:28:55.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:55 smithi190 ceph-mon[31423]: fsmap 11 up:standby 2024-04-21T10:28:55.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:55 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T10:28:55.660 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:55 smithi169 ceph-mon[31469]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:55.660 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:55 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.26:6840/1075023600,v1:172.21.15.26:6841/1075023600] up:boot 2024-04-21T10:28:55.661 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:55 smithi169 ceph-mon[31469]: fsmap 11 up:standby 2024-04-21T10:28:55.661 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:55 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T10:28:55.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:55 smithi026 ceph-mon[27965]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:55.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:55 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.26:6840/1075023600,v1:172.21.15.26:6841/1075023600] up:boot 2024-04-21T10:28:55.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:55 smithi026 ceph-mon[27965]: fsmap 11 up:standby 2024-04-21T10:28:55.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:55 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T10:28:55.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.732+0000 7fb144c6d640 1 -- 172.21.15.26:0/1489631306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb14002d5c0 con 0x7fb140102a40 2024-04-21T10:28:55.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.732+0000 7fb1377fe640 1 -- 172.21.15.26:0/1489631306 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7fb13009c5d0 con 0x7fb140102a40 2024-04-21T10:28:55.731 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:55.731 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T10:28:55.732 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 12 2024-04-21T10:28:55.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.734+0000 7fb144c6d640 1 -- 172.21.15.26:0/1489631306 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb12c0a3c30 msgr2=0x7fb12c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:55.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.734+0000 7fb144c6d640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb12c0a3c30 0x7fb12c0a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb14019b860 tx=0x7fb12800b410 comp rx=0 tx=0).stop 2024-04-21T10:28:55.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.734+0000 7fb144c6d640 1 -- 172.21.15.26:0/1489631306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140102a40 msgr2=0x7fb14019a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:55.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.734+0000 7fb144c6d640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140102a40 0x7fb14019a350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb13000d770 tx=0x7fb13000dc30 comp rx=0 tx=0).stop 2024-04-21T10:28:55.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.734+0000 7fb144c6d640 1 -- 172.21.15.26:0/1489631306 shutdown_connections 2024-04-21T10:28:55.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.734+0000 7fb144c6d640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb12c0a3c30 0x7fb12c0a60f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:55.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.734+0000 7fb144c6d640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb140107ca0 0x7fb1401a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:55.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.734+0000 7fb144c6d640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb140105370 0x7fb14019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:55.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.734+0000 7fb144c6d640 1 --2- 172.21.15.26:0/1489631306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb140102a40 0x7fb14019a350 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:55.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.734+0000 7fb144c6d640 1 -- 172.21.15.26:0/1489631306 >> 172.21.15.26:0/1489631306 conn(0x7fb1400fc470 msgr2=0x7fb1401020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:55.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.734+0000 7fb144c6d640 1 -- 172.21.15.26:0/1489631306 shutdown_connections 2024-04-21T10:28:55.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:55.734+0000 7fb144c6d640 1 -- 172.21.15.26:0/1489631306 wait complete. 2024-04-21T10:28:55.948 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:28:55 smithi169 systemd[1]: Starting Ceph mds.b for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:28:56.322 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:56 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1489631306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:56.322 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:28:56 smithi169 podman[62125]: 2024-04-21 10:28:55.964545737 +0000 UTC m=+0.018256138 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:28:56.323 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:28:56 smithi169 podman[62125]: 2024-04-21 10:28:56.087194357 +0000 UTC m=+0.140904788 container create 9b7427cf3b1152abb1082d43a31939579a00721a180ac49f4dbb87939ffa73d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-21T10:28:56.323 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:28:56 smithi169 podman[62125]: 2024-04-21 10:28:56.297604281 +0000 UTC m=+0.351314670 container init 9b7427cf3b1152abb1082d43a31939579a00721a180ac49f4dbb87939ffa73d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:28:56.323 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:28:56 smithi169 podman[62125]: 2024-04-21 10:28:56.300454967 +0000 UTC m=+0.354165360 container start 9b7427cf3b1152abb1082d43a31939579a00721a180ac49f4dbb87939ffa73d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:28:56.323 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:28:56 smithi169 ceph-mds[62145]: -- 172.21.15.169:0/3982003266 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a74ac5a40 con 0x561a74a99000 2024-04-21T10:28:56.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:56 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1489631306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:56.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:56 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1489631306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:56.698 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:28:56 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b[62141]: starting mds.b at 2024-04-21T10:28:56.698 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:28:56 smithi169 bash[62125]: 9b7427cf3b1152abb1082d43a31939579a00721a180ac49f4dbb87939ffa73d9 2024-04-21T10:28:56.698 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:28:56 smithi169 systemd[1]: Started Ceph mds.b for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:28:57.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:57 smithi026 ceph-mon[27965]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:57.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:57 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:57 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:57 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:57 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:57 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:57 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:57.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:57 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:57 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] up:boot 2024-04-21T10:28:57.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:57 smithi026 ceph-mon[27965]: fsmap 12 up:standby 2024-04-21T10:28:57.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:57 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T10:28:57.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:57 smithi169 ceph-mon[31469]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:57.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:57 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:57 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:57 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:57 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:57 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:57 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:57.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:57 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:57 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] up:boot 2024-04-21T10:28:57.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:57 smithi169 ceph-mon[31469]: fsmap 12 up:standby 2024-04-21T10:28:57.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:57 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T10:28:57.745 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:28:57.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:57 smithi190 ceph-mon[31423]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:57.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:57 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:57 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:57 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:57 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:57 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:57 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:57.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:57 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:57.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:57 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] up:boot 2024-04-21T10:28:57.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:57 smithi190 ceph-mon[31423]: fsmap 12 up:standby 2024-04-21T10:28:57.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:57 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T10:28:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.852+0000 7f4f3586d640 1 -- 172.21.15.26:0/2637689642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100870 msgr2=0x7f4f30100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.852+0000 7f4f3586d640 1 --2- 172.21.15.26:0/2637689642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100870 0x7f4f30100c90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4f20009ca0 tx=0x7f4f2002c180 comp rx=0 tx=0).stop 2024-04-21T10:28:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.852+0000 7f4f3586d640 1 -- 172.21.15.26:0/2637689642 shutdown_connections 2024-04-21T10:28:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.852+0000 7f4f3586d640 1 --2- 172.21.15.26:0/2637689642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f30101b70 0x7f4f30101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.852+0000 7f4f3586d640 1 --2- 172.21.15.26:0/2637689642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f301011d0 0x7f4f30101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.852+0000 7f4f3586d640 1 --2- 172.21.15.26:0/2637689642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100870 0x7f4f30100c90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.852+0000 7f4f3586d640 1 -- 172.21.15.26:0/2637689642 >> 172.21.15.26:0/2637689642 conn(0x7f4f300fc410 msgr2=0x7f4f300fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.852+0000 7f4f3586d640 1 -- 172.21.15.26:0/2637689642 shutdown_connections 2024-04-21T10:28:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.852+0000 7f4f3586d640 1 -- 172.21.15.26:0/2637689642 wait complete. 2024-04-21T10:28:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.852+0000 7f4f3586d640 1 Processor -- start 2024-04-21T10:28:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f3586d640 1 -- start start 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f3586d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100870 0x7f4f3019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f3586d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f301011d0 0x7f4f3019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f3586d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f30101b70 0x7f4f301a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f3586d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3010fcc0 con 0x7f4f30100870 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f3586d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3010fb40 con 0x7f4f301011d0 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f3586d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3010fe40 con 0x7f4f30101b70 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f2effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100870 0x7f4f3019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f2effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100870 0x7f4f3019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58534/0 (socket says 172.21.15.26:58534) 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f2effd640 1 -- 172.21.15.26:0/2754579864 learned_addr learned my addr 172.21.15.26:0/2754579864 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f2e7fc640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f301011d0 0x7f4f3019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f2f7fe640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f30101b70 0x7f4f301a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f2effd640 1 -- 172.21.15.26:0/2754579864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f30101b70 msgr2=0x7f4f301a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f2effd640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f30101b70 0x7f4f301a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:57.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f2effd640 1 -- 172.21.15.26:0/2754579864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f301011d0 msgr2=0x7f4f3019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:57.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f2effd640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f301011d0 0x7f4f3019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:57.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f2effd640 1 -- 172.21.15.26:0/2754579864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f301a1ff0 con 0x7f4f30100870 2024-04-21T10:28:57.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.853+0000 7f4f2effd640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100870 0x7f4f3019a3d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4f20005cc0 tx=0x7f4f20003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:57.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.854+0000 7f4f3486b640 1 -- 172.21.15.26:0/2754579864 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f20039040 con 0x7f4f30100870 2024-04-21T10:28:57.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.854+0000 7f4f3486b640 1 -- 172.21.15.26:0/2754579864 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f20035e20 con 0x7f4f30100870 2024-04-21T10:28:57.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.854+0000 7f4f3586d640 1 -- 172.21.15.26:0/2754579864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f301a2280 con 0x7f4f30100870 2024-04-21T10:28:57.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.854+0000 7f4f3486b640 1 -- 172.21.15.26:0/2754579864 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f20032b10 con 0x7f4f30100870 2024-04-21T10:28:57.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.854+0000 7f4f3586d640 1 -- 172.21.15.26:0/2754579864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f301a2770 con 0x7f4f30100870 2024-04-21T10:28:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.855+0000 7f4f3586d640 1 -- 172.21.15.26:0/2754579864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3010b810 con 0x7f4f30100870 2024-04-21T10:28:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.856+0000 7f4f3486b640 1 -- 172.21.15.26:0/2754579864 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4f20046030 con 0x7f4f30100870 2024-04-21T10:28:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.856+0000 7f4f3486b640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4f180a3bb0 0x7f4f180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.856+0000 7f4f2e7fc640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4f180a3bb0 0x7f4f180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.856+0000 7f4f3486b640 1 -- 172.21.15.26:0/2754579864 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f4f200f1b90 con 0x7f4f30100870 2024-04-21T10:28:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.857+0000 7f4f2e7fc640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4f180a3bb0 0x7f4f180a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4f3019b8e0 tx=0x7f4f2400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:57.861+0000 7f4f3486b640 1 -- 172.21.15.26:0/2754579864 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f2004d060 con 0x7f4f30100870 2024-04-21T10:28:58.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.106+0000 7f4f3586d640 1 -- 172.21.15.26:0/2754579864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f3002d5c0 con 0x7f4f30100870 2024-04-21T10:28:58.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.107+0000 7f4f3486b640 1 -- 172.21.15.26:0/2754579864 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10497 (secure 0 0 0) 0x7f4f200beaa0 con 0x7f4f30100870 2024-04-21T10:28:58.106 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:58.106 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T10:28:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 13 2024-04-21T10:28:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.110+0000 7f4f3586d640 1 -- 172.21.15.26:0/2754579864 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4f180a3bb0 msgr2=0x7f4f180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.110+0000 7f4f3586d640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4f180a3bb0 0x7f4f180a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4f3019b8e0 tx=0x7f4f2400b390 comp rx=0 tx=0).stop 2024-04-21T10:28:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.110+0000 7f4f3586d640 1 -- 172.21.15.26:0/2754579864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100870 msgr2=0x7f4f3019a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.110+0000 7f4f3586d640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100870 0x7f4f3019a3d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4f20005cc0 tx=0x7f4f20003c20 comp rx=0 tx=0).stop 2024-04-21T10:28:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.110+0000 7f4f3586d640 1 -- 172.21.15.26:0/2754579864 shutdown_connections 2024-04-21T10:28:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.110+0000 7f4f3586d640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4f180a3bb0 0x7f4f180a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.110+0000 7f4f3586d640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f30101b70 0x7f4f301a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.110+0000 7f4f3586d640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f301011d0 0x7f4f3019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.110+0000 7f4f3586d640 1 --2- 172.21.15.26:0/2754579864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f30100870 0x7f4f3019a3d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.110+0000 7f4f3586d640 1 -- 172.21.15.26:0/2754579864 >> 172.21.15.26:0/2754579864 conn(0x7f4f300fc410 msgr2=0x7f4f300fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:58.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.110+0000 7f4f3586d640 1 -- 172.21.15.26:0/2754579864 shutdown_connections 2024-04-21T10:28:58.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.110+0000 7f4f3586d640 1 -- 172.21.15.26:0/2754579864 wait complete. 2024-04-21T10:28:58.120 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:28:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.230+0000 7f3870b15640 1 -- 172.21.15.26:0/3553052328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f386c103190 msgr2=0x7f386c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.230+0000 7f3870b15640 1 --2- 172.21.15.26:0/3553052328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f386c103190 0x7f386c103610 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3860009900 tx=0x7f386002bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.230+0000 7f3870b15640 1 -- 172.21.15.26:0/3553052328 shutdown_connections 2024-04-21T10:28:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.230+0000 7f3870b15640 1 --2- 172.21.15.26:0/3553052328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f386c103b50 0x7f386c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.230+0000 7f3870b15640 1 --2- 172.21.15.26:0/3553052328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f386c103190 0x7f386c103610 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.230+0000 7f3870b15640 1 --2- 172.21.15.26:0/3553052328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f386c101fa0 0x7f386c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.230+0000 7f3870b15640 1 -- 172.21.15.26:0/3553052328 >> 172.21.15.26:0/3553052328 conn(0x7f386c0fd530 msgr2=0x7f386c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.230+0000 7f3870b15640 1 -- 172.21.15.26:0/3553052328 shutdown_connections 2024-04-21T10:28:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.230+0000 7f3870b15640 1 -- 172.21.15.26:0/3553052328 wait complete. 2024-04-21T10:28:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f3870b15640 1 Processor -- start 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f3870b15640 1 -- start start 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f3870b15640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f386c101fa0 0x7f386c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f3870b15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f386c103190 0x7f386c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f3870b15640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f386c103b50 0x7f386c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f3870b15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f386c10dae0 con 0x7f386c103190 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f3870b15640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f386c10d960 con 0x7f386c103b50 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f3870b15640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f386c10dc60 con 0x7f386c101fa0 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f3869d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f386c103190 0x7f386c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f3869d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f386c103190 0x7f386c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58548/0 (socket says 172.21.15.26:58548) 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f3869d9b640 1 -- 172.21.15.26:0/4261826742 learned_addr learned my addr 172.21.15.26:0/4261826742 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f386a59c640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f386c101fa0 0x7f386c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.231+0000 7f386ad9d640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f386c103b50 0x7f386c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f386a59c640 1 -- 172.21.15.26:0/4261826742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f386c103b50 msgr2=0x7f386c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f386a59c640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f386c103b50 0x7f386c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f386a59c640 1 -- 172.21.15.26:0/4261826742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f386c103190 msgr2=0x7f386c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f386a59c640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f386c103190 0x7f386c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f386a59c640 1 -- 172.21.15.26:0/4261826742 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386c19fde0 con 0x7f386c101fa0 2024-04-21T10:28:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f386ad9d640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f386c103b50 0x7f386c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:28:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f3869d9b640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f386c103190 0x7f386c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:28:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f386a59c640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f386c101fa0 0x7f386c1981c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f385c00e720 tx=0x7f385c00ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f385b7fe640 1 -- 172.21.15.26:0/4261826742 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c004220 con 0x7f386c101fa0 2024-04-21T10:28:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f3870b15640 1 -- 172.21.15.26:0/4261826742 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386c1a00d0 con 0x7f386c101fa0 2024-04-21T10:28:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f385b7fe640 1 -- 172.21.15.26:0/4261826742 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f385c0043c0 con 0x7f386c101fa0 2024-04-21T10:28:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f385b7fe640 1 -- 172.21.15.26:0/4261826742 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c004a00 con 0x7f386c101fa0 2024-04-21T10:28:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.232+0000 7f3870b15640 1 -- 172.21.15.26:0/4261826742 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f386c1a05d0 con 0x7f386c101fa0 2024-04-21T10:28:58.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.233+0000 7f3870b15640 1 -- 172.21.15.26:0/4261826742 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f386c109330 con 0x7f386c101fa0 2024-04-21T10:28:58.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.234+0000 7f385b7fe640 1 -- 172.21.15.26:0/4261826742 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f385c004ba0 con 0x7f386c101fa0 2024-04-21T10:28:58.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.235+0000 7f385b7fe640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f38540a3bb0 0x7f38540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:58.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.235+0000 7f385b7fe640 1 -- 172.21.15.26:0/4261826742 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f385c0cff80 con 0x7f386c101fa0 2024-04-21T10:28:58.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.235+0000 7f3869d9b640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f38540a3bb0 0x7f38540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:58.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.235+0000 7f3869d9b640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f38540a3bb0 0x7f38540a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f386c1996d0 tx=0x7f386002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:58.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.238+0000 7f385b7fe640 1 -- 172.21.15.26:0/4261826742 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f385c09ce00 con 0x7f386c101fa0 2024-04-21T10:28:58.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.459+0000 7f3870b15640 1 -- 172.21.15.26:0/4261826742 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f386c062b50 con 0x7f386c101fa0 2024-04-21T10:28:58.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.461+0000 7f385b7fe640 1 -- 172.21.15.26:0/4261826742 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18444 (secure 0 0 0) 0x7f385c005670 con 0x7f386c101fa0 2024-04-21T10:28:58.460 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:58.460 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:28:13.521135+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:28:58.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.463+0000 7f3870b15640 1 -- 172.21.15.26:0/4261826742 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f38540a3bb0 msgr2=0x7f38540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.463+0000 7f3870b15640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f38540a3bb0 0x7f38540a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f386c1996d0 tx=0x7f386002c7e0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.463+0000 7f3870b15640 1 -- 172.21.15.26:0/4261826742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f386c101fa0 msgr2=0x7f386c1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.463+0000 7f3870b15640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f386c101fa0 0x7f386c1981c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f385c00e720 tx=0x7f385c00ebe0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.463+0000 7f3870b15640 1 -- 172.21.15.26:0/4261826742 shutdown_connections 2024-04-21T10:28:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.463+0000 7f3870b15640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f38540a3bb0 0x7f38540a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.463+0000 7f3870b15640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f386c103b50 0x7f386c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.463+0000 7f3870b15640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f386c103190 0x7f386c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.463+0000 7f3870b15640 1 --2- 172.21.15.26:0/4261826742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f386c101fa0 0x7f386c1981c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.463+0000 7f3870b15640 1 -- 172.21.15.26:0/4261826742 >> 172.21.15.26:0/4261826742 conn(0x7f386c0fd530 msgr2=0x7f386c106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:58.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.463+0000 7f3870b15640 1 -- 172.21.15.26:0/4261826742 shutdown_connections 2024-04-21T10:28:58.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.463+0000 7f3870b15640 1 -- 172.21.15.26:0/4261826742 wait complete. 2024-04-21T10:28:58.472 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T10:24:28.176891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T10:28:58.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T10:28:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.578+0000 7f3cd3677640 1 -- 172.21.15.26:0/1904109555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ccc102ad0 msgr2=0x7f3ccc104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.578+0000 7f3cd3677640 1 --2- 172.21.15.26:0/1904109555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ccc102ad0 0x7f3ccc104ec0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009900 tx=0x7f3cc802bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.578+0000 7f3cd3677640 1 -- 172.21.15.26:0/1904109555 shutdown_connections 2024-04-21T10:28:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.578+0000 7f3cd3677640 1 --2- 172.21.15.26:0/1904109555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ccc107d30 0x7f3ccc10a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.578+0000 7f3cd3677640 1 --2- 172.21.15.26:0/1904109555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ccc105400 0x7f3ccc1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd3677640 1 --2- 172.21.15.26:0/1904109555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ccc102ad0 0x7f3ccc104ec0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd3677640 1 -- 172.21.15.26:0/1904109555 >> 172.21.15.26:0/1904109555 conn(0x7f3ccc0fc400 msgr2=0x7f3ccc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd3677640 1 -- 172.21.15.26:0/1904109555 shutdown_connections 2024-04-21T10:28:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd3677640 1 -- 172.21.15.26:0/1904109555 wait complete. 2024-04-21T10:28:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd3677640 1 Processor -- start 2024-04-21T10:28:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd3677640 1 -- start start 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd3677640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ccc102ad0 0x7f3ccc198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd3677640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ccc105400 0x7f3ccc1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd3677640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ccc107d30 0x7f3ccc19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd3677640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc10db60 con 0x7f3ccc102ad0 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd3677640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc10d9e0 con 0x7f3ccc105400 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd3677640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc10dce0 con 0x7f3ccc107d30 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd1413640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ccc102ad0 0x7f3ccc198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.579+0000 7f3cd1413640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ccc102ad0 0x7f3ccc198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58564/0 (socket says 172.21.15.26:58564) 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cd1413640 1 -- 172.21.15.26:0/1158871773 learned_addr learned my addr 172.21.15.26:0/1158871773 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cd1c14640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ccc107d30 0x7f3ccc19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cd0c12640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ccc105400 0x7f3ccc1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cd1413640 1 -- 172.21.15.26:0/1158871773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ccc107d30 msgr2=0x7f3ccc19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cd1413640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ccc107d30 0x7f3ccc19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cd1413640 1 -- 172.21.15.26:0/1158871773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ccc105400 msgr2=0x7f3ccc1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cd1413640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ccc105400 0x7f3ccc1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cd1413640 1 -- 172.21.15.26:0/1158871773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ccc19fe80 con 0x7f3ccc102ad0 2024-04-21T10:28:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cd0c12640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ccc105400 0x7f3ccc1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:28:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cd1413640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ccc102ad0 0x7f3ccc198260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8006010 tx=0x7f3cc80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cc27fc640 1 -- 172.21.15.26:0/1158871773 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8043020 con 0x7f3ccc102ad0 2024-04-21T10:28:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cc27fc640 1 -- 172.21.15.26:0/1158871773 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc8004aa0 con 0x7f3ccc102ad0 2024-04-21T10:28:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cd3677640 1 -- 172.21.15.26:0/1158871773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ccc1a0110 con 0x7f3ccc102ad0 2024-04-21T10:28:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cc27fc640 1 -- 172.21.15.26:0/1158871773 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8003c70 con 0x7f3ccc102ad0 2024-04-21T10:28:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.580+0000 7f3cd3677640 1 -- 172.21.15.26:0/1158871773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ccc1a06e0 con 0x7f3ccc102ad0 2024-04-21T10:28:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.581+0000 7f3cd3677640 1 -- 172.21.15.26:0/1158871773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ccc0fdef0 con 0x7f3ccc102ad0 2024-04-21T10:28:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.582+0000 7f3cc27fc640 1 -- 172.21.15.26:0/1158871773 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3cc8046020 con 0x7f3ccc102ad0 2024-04-21T10:28:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.583+0000 7f3cc27fc640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3cb80a3bb0 0x7f3cb80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.583+0000 7f3cd0c12640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3cb80a3bb0 0x7f3cb80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.583+0000 7f3cc27fc640 1 -- 172.21.15.26:0/1158871773 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f3cc80f2390 con 0x7f3ccc102ad0 2024-04-21T10:28:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.583+0000 7f3cd0c12640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3cb80a3bb0 0x7f3cb80a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc199770 tx=0x7f3cbc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.588+0000 7f3cc27fc640 1 -- 172.21.15.26:0/1158871773 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cc804d020 con 0x7f3ccc102ad0 2024-04-21T10:28:58.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:58 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2754579864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:58.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:58 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:58 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2754579864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:58 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.773+0000 7f3cd3677640 1 -- 172.21.15.26:0/1158871773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ccc062b50 con 0x7f3ccc102ad0 2024-04-21T10:28:58.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.774+0000 7f3cc27fc640 1 -- 172.21.15.26:0/1158871773 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f3cc80bf170 con 0x7f3ccc102ad0 2024-04-21T10:28:58.773 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-21T10:28:58.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.776+0000 7f3cd3677640 1 -- 172.21.15.26:0/1158871773 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3cb80a3bb0 msgr2=0x7f3cb80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.776+0000 7f3cd3677640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3cb80a3bb0 0x7f3cb80a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc199770 tx=0x7f3cbc008040 comp rx=0 tx=0).stop 2024-04-21T10:28:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.776+0000 7f3cd3677640 1 -- 172.21.15.26:0/1158871773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ccc102ad0 msgr2=0x7f3ccc198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.776+0000 7f3cd3677640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ccc102ad0 0x7f3ccc198260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8006010 tx=0x7f3cc80042f0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.776+0000 7f3cd3677640 1 -- 172.21.15.26:0/1158871773 shutdown_connections 2024-04-21T10:28:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.776+0000 7f3cd3677640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3cb80a3bb0 0x7f3cb80a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.776+0000 7f3cd3677640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ccc107d30 0x7f3ccc19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.776+0000 7f3cd3677640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ccc105400 0x7f3ccc1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.776+0000 7f3cd3677640 1 --2- 172.21.15.26:0/1158871773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ccc102ad0 0x7f3ccc198260 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.776+0000 7f3cd3677640 1 -- 172.21.15.26:0/1158871773 >> 172.21.15.26:0/1158871773 conn(0x7f3ccc0fc400 msgr2=0x7f3ccc103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.776+0000 7f3cd3677640 1 -- 172.21.15.26:0/1158871773 shutdown_connections 2024-04-21T10:28:58.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.776+0000 7f3cd3677640 1 -- 172.21.15.26:0/1158871773 wait complete. 2024-04-21T10:28:58.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:58 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2754579864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:28:58.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:58 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:58.786 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-21T10:28:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.889+0000 7f1786175640 1 -- 172.21.15.26:0/3559881568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1780100e60 msgr2=0x7f1780101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.889+0000 7f1786175640 1 --2- 172.21.15.26:0/3559881568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1780100e60 0x7f1780101280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1770009900 tx=0x7f177002bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.889+0000 7f1786175640 1 -- 172.21.15.26:0/3559881568 shutdown_connections 2024-04-21T10:28:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.889+0000 7f1786175640 1 --2- 172.21.15.26:0/3559881568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1780102a00 0x7f1780109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.889+0000 7f1786175640 1 --2- 172.21.15.26:0/3559881568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1780102040 0x7f17801024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.889+0000 7f1786175640 1 --2- 172.21.15.26:0/3559881568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1780100e60 0x7f1780101280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.889+0000 7f1786175640 1 -- 172.21.15.26:0/3559881568 >> 172.21.15.26:0/3559881568 conn(0x7f17800fc440 msgr2=0x7f17800fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.889+0000 7f1786175640 1 -- 172.21.15.26:0/3559881568 shutdown_connections 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.889+0000 7f1786175640 1 -- 172.21.15.26:0/3559881568 wait complete. 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f1786175640 1 Processor -- start 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f1786175640 1 -- start start 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f1786175640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1780100e60 0x7f1780198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f1786175640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1780102040 0x7f1780198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f1786175640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1780102a00 0x7f178019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f1786175640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f178010db50 con 0x7f1780100e60 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f1786175640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f178010d9d0 con 0x7f1780102a00 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f1786175640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f178010dcd0 con 0x7f1780102040 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f177f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1780100e60 0x7f1780198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f177f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1780100e60 0x7f1780198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58582/0 (socket says 172.21.15.26:58582) 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f177f7fe640 1 -- 172.21.15.26:0/3931441814 learned_addr learned my addr 172.21.15.26:0/3931441814 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f177effd640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1780102040 0x7f1780198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f177ffff640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1780102a00 0x7f178019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f177f7fe640 1 -- 172.21.15.26:0/3931441814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1780102040 msgr2=0x7f1780198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f177f7fe640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1780102040 0x7f1780198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f177f7fe640 1 -- 172.21.15.26:0/3931441814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1780102a00 msgr2=0x7f178019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f177f7fe640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1780102a00 0x7f178019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.890+0000 7f177f7fe640 1 -- 172.21.15.26:0/3931441814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178019fe50 con 0x7f1780100e60 2024-04-21T10:28:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.891+0000 7f177f7fe640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1780100e60 0x7f1780198230 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1770005ab0 tx=0x7f17700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.891+0000 7f177cff9640 1 -- 172.21.15.26:0/3931441814 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770043020 con 0x7f1780100e60 2024-04-21T10:28:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.891+0000 7f1786175640 1 -- 172.21.15.26:0/3931441814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17801a00e0 con 0x7f1780100e60 2024-04-21T10:28:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.891+0000 7f177cff9640 1 -- 172.21.15.26:0/3931441814 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17700054c0 con 0x7f1780100e60 2024-04-21T10:28:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.891+0000 7f177cff9640 1 -- 172.21.15.26:0/3931441814 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770003d10 con 0x7f1780100e60 2024-04-21T10:28:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.891+0000 7f1786175640 1 -- 172.21.15.26:0/3931441814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17801a0680 con 0x7f1780100e60 2024-04-21T10:28:58.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.892+0000 7f1786175640 1 -- 172.21.15.26:0/3931441814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17801082d0 con 0x7f1780100e60 2024-04-21T10:28:58.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.893+0000 7f177cff9640 1 -- 172.21.15.26:0/3931441814 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1770046020 con 0x7f1780100e60 2024-04-21T10:28:58.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.894+0000 7f177cff9640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f17680a3bb0 0x7f17680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:58.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.894+0000 7f177effd640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f17680a3bb0 0x7f17680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:58.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.894+0000 7f177cff9640 1 -- 172.21.15.26:0/3931441814 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f17700f2770 con 0x7f1780100e60 2024-04-21T10:28:58.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.894+0000 7f177effd640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f17680a3bb0 0x7f17680a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1780199740 tx=0x7f177400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:58.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:58.897+0000 7f177cff9640 1 -- 172.21.15.26:0/3931441814 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f177004d020 con 0x7f1780100e60 2024-04-21T10:28:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.089+0000 7f1786175640 1 -- 172.21.15.26:0/3931441814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f178002d790 con 0x7f1780100e60 2024-04-21T10:28:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.090+0000 7f177cff9640 1 -- 172.21.15.26:0/3931441814 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7f17700bf550 con 0x7f1780100e60 2024-04-21T10:28:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:28:59.089 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-21T10:28:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:28:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-04-21T10:28:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-04-21T10:28:59.089 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:28:59.089 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-21T10:28:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.092+0000 7f1786175640 1 -- 172.21.15.26:0/3931441814 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f17680a3bb0 msgr2=0x7f17680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.092+0000 7f1786175640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f17680a3bb0 0x7f17680a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1780199740 tx=0x7f177400b410 comp rx=0 tx=0).stop 2024-04-21T10:28:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.092+0000 7f1786175640 1 -- 172.21.15.26:0/3931441814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1780100e60 msgr2=0x7f1780198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.092+0000 7f1786175640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1780100e60 0x7f1780198230 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1770005ab0 tx=0x7f17700042f0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.092+0000 7f1786175640 1 -- 172.21.15.26:0/3931441814 shutdown_connections 2024-04-21T10:28:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.092+0000 7f1786175640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f17680a3bb0 0x7f17680a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.092+0000 7f1786175640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1780102a00 0x7f178019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.092+0000 7f1786175640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1780102040 0x7f1780198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.092+0000 7f1786175640 1 --2- 172.21.15.26:0/3931441814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1780100e60 0x7f1780198230 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.092+0000 7f1786175640 1 -- 172.21.15.26:0/3931441814 >> 172.21.15.26:0/3931441814 conn(0x7f17800fc440 msgr2=0x7f17801054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.092+0000 7f1786175640 1 -- 172.21.15.26:0/3931441814 shutdown_connections 2024-04-21T10:28:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.092+0000 7f1786175640 1 -- 172.21.15.26:0/3931441814 wait complete. 2024-04-21T10:28:59.102 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-21T10:28:59.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-21T10:28:59.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.213+0000 7f0e940d9640 1 -- 172.21.15.26:0/2927109566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e8c100e60 msgr2=0x7f0e8c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:59.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.213+0000 7f0e940d9640 1 --2- 172.21.15.26:0/2927109566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e8c100e60 0x7f0e8c101280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009900 tx=0x7f0e8802bc30 comp rx=0 tx=0).stop 2024-04-21T10:28:59.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.213+0000 7f0e940d9640 1 -- 172.21.15.26:0/2927109566 shutdown_connections 2024-04-21T10:28:59.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.213+0000 7f0e940d9640 1 --2- 172.21.15.26:0/2927109566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e8c102a00 0x7f0e8c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.213+0000 7f0e940d9640 1 --2- 172.21.15.26:0/2927109566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e8c102040 0x7f0e8c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.213+0000 7f0e940d9640 1 --2- 172.21.15.26:0/2927109566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e8c100e60 0x7f0e8c101280 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.213+0000 7f0e940d9640 1 -- 172.21.15.26:0/2927109566 >> 172.21.15.26:0/2927109566 conn(0x7f0e8c0fc420 msgr2=0x7f0e8c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e940d9640 1 -- 172.21.15.26:0/2927109566 shutdown_connections 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e940d9640 1 -- 172.21.15.26:0/2927109566 wait complete. 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e940d9640 1 Processor -- start 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e940d9640 1 -- start start 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e940d9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e8c100e60 0x7f0e8c104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e940d9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e8c102040 0x7f0e8c1034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e940d9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e8c102a00 0x7f0e8c1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e940d9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c10ca40 con 0x7f0e8c100e60 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e940d9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c10c8c0 con 0x7f0e8c102a00 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e940d9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c10cbc0 con 0x7f0e8c102040 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e91e75640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e8c100e60 0x7f0e8c104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:59.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e91e75640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e8c100e60 0x7f0e8c104e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58604/0 (socket says 172.21.15.26:58604) 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e91e75640 1 -- 172.21.15.26:0/968038193 learned_addr learned my addr 172.21.15.26:0/968038193 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e91674640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e8c102040 0x7f0e8c1034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.214+0000 7f0e92676640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e8c102a00 0x7f0e8c1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.215+0000 7f0e91e75640 1 -- 172.21.15.26:0/968038193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e8c102040 msgr2=0x7f0e8c1034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.215+0000 7f0e91e75640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e8c102040 0x7f0e8c1034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.215+0000 7f0e91e75640 1 -- 172.21.15.26:0/968038193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e8c102a00 msgr2=0x7f0e8c1039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.215+0000 7f0e91e75640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e8c102a00 0x7f0e8c1039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.215+0000 7f0e91e75640 1 -- 172.21.15.26:0/968038193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c104280 con 0x7f0e8c100e60 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.215+0000 7f0e92676640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e8c102a00 0x7f0e8c1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.215+0000 7f0e91e75640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e8c100e60 0x7f0e8c104e70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e88000c00 tx=0x7f0e88005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.215+0000 7f0e82ffd640 1 -- 172.21.15.26:0/968038193 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88043020 con 0x7f0e8c100e60 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.215+0000 7f0e940d9640 1 -- 172.21.15.26:0/968038193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8c1a2360 con 0x7f0e8c100e60 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.215+0000 7f0e82ffd640 1 -- 172.21.15.26:0/968038193 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e880043d0 con 0x7f0e8c100e60 2024-04-21T10:28:59.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.215+0000 7f0e940d9640 1 -- 172.21.15.26:0/968038193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8c1a2900 con 0x7f0e8c100e60 2024-04-21T10:28:59.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.215+0000 7f0e82ffd640 1 -- 172.21.15.26:0/968038193 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88003b20 con 0x7f0e8c100e60 2024-04-21T10:28:59.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.216+0000 7f0e940d9640 1 -- 172.21.15.26:0/968038193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8c1a2c20 con 0x7f0e8c100e60 2024-04-21T10:28:59.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.217+0000 7f0e82ffd640 1 -- 172.21.15.26:0/968038193 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0e88046020 con 0x7f0e8c100e60 2024-04-21T10:28:59.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.218+0000 7f0e82ffd640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0e780a3bb0 0x7f0e780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:28:59.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.218+0000 7f0e91674640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0e780a3bb0 0x7f0e780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:28:59.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.218+0000 7f0e82ffd640 1 -- 172.21.15.26:0/968038193 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f0e880f2130 con 0x7f0e8c100e60 2024-04-21T10:28:59.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.219+0000 7f0e91674640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0e780a3bb0 0x7f0e780a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c104be0 tx=0x7f0e7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:28:59.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.223+0000 7f0e82ffd640 1 -- 172.21.15.26:0/968038193 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e8804d020 con 0x7f0e8c100e60 2024-04-21T10:28:59.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.413+0000 7f0e940d9640 1 -- 172.21.15.26:0/968038193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0e8c1a2df0 con 0x7f0e8c100e60 2024-04-21T10:28:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:59 smithi169 ceph-mon[31469]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:59 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/4261826742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:28:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:59 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1158871773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:28:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:59 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:59 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:28:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:59 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:59 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:59 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3931441814' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T10:28:59.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:28:59 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/968038193' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T10:28:59.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:59 smithi190 ceph-mon[31423]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:59.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:59 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/4261826742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:28:59.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:59 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1158871773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:28:59.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:59.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:28:59.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:59.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:59 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:59.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:59 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3931441814' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T10:28:59.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:28:59 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/968038193' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T10:28:59.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:59 smithi026 ceph-mon[27965]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:28:59.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:59 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/4261826742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:28:59.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:59 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1158871773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:28:59.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:28:59.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:28:59.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:28:59.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:59 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:28:59.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:59 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3931441814' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T10:28:59.882 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:28:59 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/968038193' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T10:28:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.903+0000 7f0e82ffd640 1 -- 172.21.15.26:0/968038193 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f0e880bef10 con 0x7f0e8c100e60 2024-04-21T10:28:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-04-21T10:28:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.906+0000 7f0e940d9640 1 -- 172.21.15.26:0/968038193 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0e780a3bb0 msgr2=0x7f0e780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.906+0000 7f0e940d9640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0e780a3bb0 0x7f0e780a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c104be0 tx=0x7f0e7c005f90 comp rx=0 tx=0).stop 2024-04-21T10:28:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.906+0000 7f0e940d9640 1 -- 172.21.15.26:0/968038193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e8c100e60 msgr2=0x7f0e8c104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:28:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.906+0000 7f0e940d9640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e8c100e60 0x7f0e8c104e70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e88000c00 tx=0x7f0e88005e60 comp rx=0 tx=0).stop 2024-04-21T10:28:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.906+0000 7f0e940d9640 1 -- 172.21.15.26:0/968038193 shutdown_connections 2024-04-21T10:28:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.906+0000 7f0e940d9640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0e780a3bb0 0x7f0e780a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.906+0000 7f0e940d9640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e8c102a00 0x7f0e8c1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.906+0000 7f0e940d9640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e8c102040 0x7f0e8c1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.906+0000 7f0e940d9640 1 --2- 172.21.15.26:0/968038193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e8c100e60 0x7f0e8c104e70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:28:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.906+0000 7f0e940d9640 1 -- 172.21.15.26:0/968038193 >> 172.21.15.26:0/968038193 conn(0x7f0e8c0fc420 msgr2=0x7f0e8c06a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:28:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.906+0000 7f0e940d9640 1 -- 172.21.15.26:0/968038193 shutdown_connections 2024-04-21T10:28:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:28:59.906+0000 7f0e940d9640 1 -- 172.21.15.26:0/968038193 wait complete. 2024-04-21T10:28:59.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-21T10:29:00.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.034+0000 7f591dda4640 1 -- 172.21.15.26:0/440921911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5918102a10 msgr2=0x7f5918104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:00.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.034+0000 7f591dda4640 1 --2- 172.21.15.26:0/440921911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5918102a10 0x7f5918104e00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5908009900 tx=0x7f590802bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:00.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.034+0000 7f591dda4640 1 -- 172.21.15.26:0/440921911 shutdown_connections 2024-04-21T10:29:00.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.034+0000 7f591dda4640 1 --2- 172.21.15.26:0/440921911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5918107c70 0x7f591810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.034+0000 7f591dda4640 1 --2- 172.21.15.26:0/440921911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5918105340 0x7f5918107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.034+0000 7f591dda4640 1 --2- 172.21.15.26:0/440921911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5918102a10 0x7f5918104e00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.034+0000 7f591dda4640 1 -- 172.21.15.26:0/440921911 >> 172.21.15.26:0/440921911 conn(0x7f59180fc400 msgr2=0x7f59180fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.034+0000 7f591dda4640 1 -- 172.21.15.26:0/440921911 shutdown_connections 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.034+0000 7f591dda4640 1 -- 172.21.15.26:0/440921911 wait complete. 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f591dda4640 1 Processor -- start 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f591dda4640 1 -- start start 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f591dda4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5918102a10 0x7f5918198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f591dda4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5918105340 0x7f5918198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f591dda4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5918107c70 0x7f591819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f591dda4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f591810db20 con 0x7f5918107c70 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f591dda4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f591810d9a0 con 0x7f5918105340 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f591dda4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f591810dca0 con 0x7f5918102a10 2024-04-21T10:29:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f5917fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5918107c70 0x7f591819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f5917fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5918107c70 0x7f591819f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58624/0 (socket says 172.21.15.26:58624) 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f59177fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5918102a10 0x7f5918198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f5917fff640 1 -- 172.21.15.26:0/1720425560 learned_addr learned my addr 172.21.15.26:0/1720425560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f5916ffd640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5918105340 0x7f5918198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f5917fff640 1 -- 172.21.15.26:0/1720425560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5918102a10 msgr2=0x7f5918198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f5917fff640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5918102a10 0x7f5918198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f5917fff640 1 -- 172.21.15.26:0/1720425560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5918105340 msgr2=0x7f5918198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f5917fff640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5918105340 0x7f5918198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f5917fff640 1 -- 172.21.15.26:0/1720425560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591819fe70 con 0x7f5918107c70 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f5916ffd640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5918105340 0x7f5918198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.035+0000 7f59177fe640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5918102a10 0x7f5918198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.036+0000 7f5917fff640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5918107c70 0x7f591819f800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f591000eb10 tx=0x7f591000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.036+0000 7f5914ff9640 1 -- 172.21.15.26:0/1720425560 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910004b20 con 0x7f5918107c70 2024-04-21T10:29:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.036+0000 7f591dda4640 1 -- 172.21.15.26:0/1720425560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59181a0160 con 0x7f5918107c70 2024-04-21T10:29:00.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.036+0000 7f5914ff9640 1 -- 172.21.15.26:0/1720425560 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5910004cc0 con 0x7f5918107c70 2024-04-21T10:29:00.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.036+0000 7f5914ff9640 1 -- 172.21.15.26:0/1720425560 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910010670 con 0x7f5918107c70 2024-04-21T10:29:00.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.036+0000 7f591dda4640 1 -- 172.21.15.26:0/1720425560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59181a0660 con 0x7f5918107c70 2024-04-21T10:29:00.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.037+0000 7f591dda4640 1 -- 172.21.15.26:0/1720425560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59180fdef0 con 0x7f5918107c70 2024-04-21T10:29:00.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.038+0000 7f5914ff9640 1 -- 172.21.15.26:0/1720425560 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5910005000 con 0x7f5918107c70 2024-04-21T10:29:00.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.039+0000 7f5914ff9640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f59000a3bb0 0x7f59000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:00.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.039+0000 7f59177fe640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f59000a3bb0 0x7f59000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:00.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.039+0000 7f5914ff9640 1 -- 172.21.15.26:0/1720425560 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7f59100d09b0 con 0x7f5918107c70 2024-04-21T10:29:00.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.039+0000 7f59177fe640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f59000a3bb0 0x7f59000a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5908000c00 tx=0x7f590802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:00.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.042+0000 7f5914ff9640 1 -- 172.21.15.26:0/1720425560 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5910053ff0 con 0x7f5918107c70 2024-04-21T10:29:00.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.231+0000 7f591dda4640 1 -- 172.21.15.26:0/1720425560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f591802d5c0 con 0x7f5918107c70 2024-04-21T10:29:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.909+0000 7f5914ff9640 1 -- 172.21.15.26:0/1720425560 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f591009d620 con 0x7f5918107c70 2024-04-21T10:29:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-04-21T10:29:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.912+0000 7f591dda4640 1 -- 172.21.15.26:0/1720425560 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f59000a3bb0 msgr2=0x7f59000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.912+0000 7f591dda4640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f59000a3bb0 0x7f59000a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5908000c00 tx=0x7f590802c7e0 comp rx=0 tx=0).stop 2024-04-21T10:29:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.912+0000 7f591dda4640 1 -- 172.21.15.26:0/1720425560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5918107c70 msgr2=0x7f591819f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.912+0000 7f591dda4640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5918107c70 0x7f591819f800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f591000eb10 tx=0x7f591000c2d0 comp rx=0 tx=0).stop 2024-04-21T10:29:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.913+0000 7f591dda4640 1 -- 172.21.15.26:0/1720425560 shutdown_connections 2024-04-21T10:29:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.913+0000 7f591dda4640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f59000a3bb0 0x7f59000a6070 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5908000c00 tx=0x7f590802c7e0 comp rx=0 tx=0).stop 2024-04-21T10:29:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.913+0000 7f591dda4640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5918107c70 0x7f591819f800 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.913+0000 7f591dda4640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5918105340 0x7f5918198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.913+0000 7f591dda4640 1 --2- 172.21.15.26:0/1720425560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5918102a10 0x7f5918198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.913+0000 7f591dda4640 1 -- 172.21.15.26:0/1720425560 >> 172.21.15.26:0/1720425560 conn(0x7f59180fc400 msgr2=0x7f59181048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.916+0000 7f591dda4640 1 -- 172.21.15.26:0/1720425560 shutdown_connections 2024-04-21T10:29:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:00.917+0000 7f591dda4640 1 -- 172.21.15.26:0/1720425560 wait complete. 2024-04-21T10:29:00.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-21T10:29:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.041+0000 7f5494f52640 1 -- 172.21.15.26:0/967638789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5490100e60 msgr2=0x7f5490101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.041+0000 7f5494f52640 1 --2- 172.21.15.26:0/967638789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5490100e60 0x7f5490101280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5480009900 tx=0x7f548002bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:01.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.041+0000 7f5494f52640 1 -- 172.21.15.26:0/967638789 shutdown_connections 2024-04-21T10:29:01.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.041+0000 7f5494f52640 1 --2- 172.21.15.26:0/967638789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5490102a00 0x7f5490109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:01.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.041+0000 7f5494f52640 1 --2- 172.21.15.26:0/967638789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5490102040 0x7f54901024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:01.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.041+0000 7f5494f52640 1 --2- 172.21.15.26:0/967638789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5490100e60 0x7f5490101280 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:01.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.041+0000 7f5494f52640 1 -- 172.21.15.26:0/967638789 >> 172.21.15.26:0/967638789 conn(0x7f54900fc440 msgr2=0x7f54900fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:01.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.041+0000 7f5494f52640 1 -- 172.21.15.26:0/967638789 shutdown_connections 2024-04-21T10:29:01.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.041+0000 7f5494f52640 1 -- 172.21.15.26:0/967638789 wait complete. 2024-04-21T10:29:01.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f5494f52640 1 Processor -- start 2024-04-21T10:29:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f5494f52640 1 -- start start 2024-04-21T10:29:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f5494f52640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5490100e60 0x7f54901982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f5494f52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5490102040 0x7f5490198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f5494f52640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5490102a00 0x7f549019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f5494f52640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f549010cab0 con 0x7f5490102040 2024-04-21T10:29:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f5494f52640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f549010c930 con 0x7f5490102a00 2024-04-21T10:29:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f5494f52640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f549010cc30 con 0x7f5490100e60 2024-04-21T10:29:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f548dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5490102040 0x7f5490198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f548dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5490102040 0x7f5490198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58652/0 (socket says 172.21.15.26:58652) 2024-04-21T10:29:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f548dd9b640 1 -- 172.21.15.26:0/2379170487 learned_addr learned my addr 172.21.15.26:0/2379170487 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f548e59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5490100e60 0x7f54901982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.042+0000 7f548ed9d640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5490102a00 0x7f549019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.043+0000 7f548dd9b640 1 -- 172.21.15.26:0/2379170487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5490100e60 msgr2=0x7f54901982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.043+0000 7f548dd9b640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5490100e60 0x7f54901982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.043+0000 7f548dd9b640 1 -- 172.21.15.26:0/2379170487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5490102a00 msgr2=0x7f549019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.043+0000 7f548dd9b640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5490102a00 0x7f549019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.043+0000 7f548dd9b640 1 -- 172.21.15.26:0/2379170487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549019fea0 con 0x7f5490102040 2024-04-21T10:29:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.043+0000 7f548dd9b640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5490102040 0x7f5490198810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f548400d950 tx=0x7f548400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.043+0000 7f547f7fe640 1 -- 172.21.15.26:0/2379170487 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484017030 con 0x7f5490102040 2024-04-21T10:29:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.043+0000 7f5494f52640 1 -- 172.21.15.26:0/2379170487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54901a0190 con 0x7f5490102040 2024-04-21T10:29:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.043+0000 7f547f7fe640 1 -- 172.21.15.26:0/2379170487 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5484004020 con 0x7f5490102040 2024-04-21T10:29:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.043+0000 7f547f7fe640 1 -- 172.21.15.26:0/2379170487 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54840047a0 con 0x7f5490102040 2024-04-21T10:29:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.043+0000 7f5494f52640 1 -- 172.21.15.26:0/2379170487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54901a0730 con 0x7f5490102040 2024-04-21T10:29:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.044+0000 7f5494f52640 1 -- 172.21.15.26:0/2379170487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54901a09f0 con 0x7f5490102040 2024-04-21T10:29:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.046+0000 7f547f7fe640 1 -- 172.21.15.26:0/2379170487 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5484009a30 con 0x7f5490102040 2024-04-21T10:29:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.047+0000 7f547f7fe640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f54780a3bb0 0x7f54780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.047+0000 7f548e59c640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f54780a3bb0 0x7f54780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.047+0000 7f547f7fe640 1 -- 172.21.15.26:0/2379170487 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7879+0+0 (secure 0 0 0) 0x7f54840d02d0 con 0x7f5490102040 2024-04-21T10:29:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.047+0000 7f548e59c640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f54780a3bb0 0x7f54780a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f548002c240 tx=0x7f548002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:01.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.050+0000 7f547f7fe640 1 -- 172.21.15.26:0/2379170487 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f548409ce60 con 0x7f5490102040 2024-04-21T10:29:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:00 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/968038193' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T10:29:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:00 smithi026 ceph-mon[27965]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T10:29:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:00 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1720425560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T10:29:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:29:01.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:29:01.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:00 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:00 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/968038193' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T10:29:01.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:00 smithi169 ceph-mon[31469]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T10:29:01.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:00 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1720425560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T10:29:01.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:00 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:00 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:00 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:00 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:00 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:00 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:00 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:29:01.200 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:00 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:29:01.200 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:00 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:00 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/968038193' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T10:29:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:00 smithi190 ceph-mon[31423]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T10:29:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:00 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1720425560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T10:29:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:29:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:29:01.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:00 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:29:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.278+0000 7f5494f52640 1 -- 172.21.15.26:0/2379170487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f54901a0df0 con 0x7f5490102040 2024-04-21T10:29:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.911+0000 7f547f7fe640 1 -- 172.21.15.26:0/2379170487 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f548409c760 con 0x7f5490102040 2024-04-21T10:29:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 2 and data pool 3 2024-04-21T10:29:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.914+0000 7f5494f52640 1 -- 172.21.15.26:0/2379170487 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f54780a3bb0 msgr2=0x7f54780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.914+0000 7f5494f52640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f54780a3bb0 0x7f54780a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f548002c240 tx=0x7f548002c7e0 comp rx=0 tx=0).stop 2024-04-21T10:29:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.915+0000 7f5494f52640 1 -- 172.21.15.26:0/2379170487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5490102040 msgr2=0x7f5490198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.915+0000 7f5494f52640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5490102040 0x7f5490198810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f548400d950 tx=0x7f548400de10 comp rx=0 tx=0).stop 2024-04-21T10:29:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.915+0000 7f5494f52640 1 -- 172.21.15.26:0/2379170487 shutdown_connections 2024-04-21T10:29:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.915+0000 7f5494f52640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f54780a3bb0 0x7f54780a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.915+0000 7f5494f52640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5490102a00 0x7f549019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.915+0000 7f5494f52640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5490102040 0x7f5490198810 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f548400d950 tx=0x7f548400de10 comp rx=0 tx=0).stop 2024-04-21T10:29:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.915+0000 7f5494f52640 1 --2- 172.21.15.26:0/2379170487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5490100e60 0x7f54901982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:01.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.915+0000 7f5494f52640 1 -- 172.21.15.26:0/2379170487 >> 172.21.15.26:0/2379170487 conn(0x7f54900fc440 msgr2=0x7f54900fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:01.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.915+0000 7f5494f52640 1 -- 172.21.15.26:0/2379170487 shutdown_connections 2024-04-21T10:29:01.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:01.915+0000 7f5494f52640 1 -- 172.21.15.26:0/2379170487 wait complete. 2024-04-21T10:29:01.938 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:29:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.049+0000 7f60de50d640 1 -- 172.21.15.26:0/1623557026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d81053b0 msgr2=0x7f60d81077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.049+0000 7f60de50d640 1 --2- 172.21.15.26:0/1623557026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d81053b0 0x7f60d81077a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009900 tx=0x7f60cc02bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:02.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.049+0000 7f60de50d640 1 -- 172.21.15.26:0/1623557026 shutdown_connections 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.049+0000 7f60de50d640 1 --2- 172.21.15.26:0/1623557026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d8107ce0 0x7f60d810a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.049+0000 7f60de50d640 1 --2- 172.21.15.26:0/1623557026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d81053b0 0x7f60d81077a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.049+0000 7f60de50d640 1 --2- 172.21.15.26:0/1623557026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60d8102a80 0x7f60d8104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.049+0000 7f60de50d640 1 -- 172.21.15.26:0/1623557026 >> 172.21.15.26:0/1623557026 conn(0x7f60d80fc410 msgr2=0x7f60d80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.049+0000 7f60de50d640 1 -- 172.21.15.26:0/1623557026 shutdown_connections 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.049+0000 7f60de50d640 1 -- 172.21.15.26:0/1623557026 wait complete. 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60de50d640 1 Processor -- start 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60de50d640 1 -- start start 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60de50d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d8102a80 0x7f60d8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60de50d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d81053b0 0x7f60d8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60de50d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60d8107ce0 0x7f60d819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60de50d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60d810db20 con 0x7f60d8102a80 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60de50d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f60d810d9a0 con 0x7f60d81053b0 2024-04-21T10:29:02.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60de50d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f60d810dca0 con 0x7f60d8107ce0 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d8102a80 0x7f60d8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60d7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d8102a80 0x7f60d8198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58670/0 (socket says 172.21.15.26:58670) 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60d7fff640 1 -- 172.21.15.26:0/2354529515 learned_addr learned my addr 172.21.15.26:0/2354529515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60dcaaa640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60d8107ce0 0x7f60d819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.050+0000 7f60d77fe640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d81053b0 0x7f60d8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.051+0000 7f60d7fff640 1 -- 172.21.15.26:0/2354529515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60d8107ce0 msgr2=0x7f60d819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.051+0000 7f60d7fff640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60d8107ce0 0x7f60d819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.051+0000 7f60d7fff640 1 -- 172.21.15.26:0/2354529515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d81053b0 msgr2=0x7f60d8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.051+0000 7f60d7fff640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d81053b0 0x7f60d8198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.051+0000 7f60d7fff640 1 -- 172.21.15.26:0/2354529515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d819fe30 con 0x7f60d8102a80 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.051+0000 7f60d77fe640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d81053b0 0x7f60d8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.051+0000 7f60d7fff640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d8102a80 0x7f60d8198210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f60c800d950 tx=0x7f60c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.051+0000 7f60d57fa640 1 -- 172.21.15.26:0/2354529515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8017030 con 0x7f60d8102a80 2024-04-21T10:29:02.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.051+0000 7f60de50d640 1 -- 172.21.15.26:0/2354529515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d81a0120 con 0x7f60d8102a80 2024-04-21T10:29:02.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.051+0000 7f60d57fa640 1 -- 172.21.15.26:0/2354529515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60c8004020 con 0x7f60d8102a80 2024-04-21T10:29:02.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.051+0000 7f60d57fa640 1 -- 172.21.15.26:0/2354529515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c80047a0 con 0x7f60d8102a80 2024-04-21T10:29:02.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.051+0000 7f60de50d640 1 -- 172.21.15.26:0/2354529515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d81a06c0 con 0x7f60d8102a80 2024-04-21T10:29:02.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.052+0000 7f60de50d640 1 -- 172.21.15.26:0/2354529515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d80fdf00 con 0x7f60d8102a80 2024-04-21T10:29:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.053+0000 7f60d57fa640 1 -- 172.21.15.26:0/2354529515 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f60c8009a30 con 0x7f60d8102a80 2024-04-21T10:29:02.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.054+0000 7f60d57fa640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f60c00a3bb0 0x7f60c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:02.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.054+0000 7f60d77fe640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f60c00a3bb0 0x7f60c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:02.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.054+0000 7f60d57fa640 1 -- 172.21.15.26:0/2354529515 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f60c80d0410 con 0x7f60d8102a80 2024-04-21T10:29:02.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.054+0000 7f60d77fe640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f60c00a3bb0 0x7f60c00a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f60cc036040 tx=0x7f60cc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:02.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.057+0000 7f60d57fa640 1 -- 172.21.15.26:0/2354529515 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60c809cf40 con 0x7f60d8102a80 2024-04-21T10:29:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:01 smithi026 ceph-mon[27965]: pgmap v283: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:29:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:01 smithi026 ceph-mon[27965]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T10:29:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:01 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1720425560' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T10:29:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:01 smithi026 ceph-mon[27965]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T10:29:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:01 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2379170487' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T10:29:02.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:01 smithi026 ceph-mon[27965]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-21T10:29:02.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:01 smithi169 ceph-mon[31469]: pgmap v283: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:29:02.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:01 smithi169 ceph-mon[31469]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T10:29:02.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:01 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1720425560' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T10:29:02.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:01 smithi169 ceph-mon[31469]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T10:29:02.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:01 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2379170487' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T10:29:02.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:01 smithi169 ceph-mon[31469]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-21T10:29:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.248+0000 7f60de50d640 1 -- 172.21.15.26:0/2354529515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60d8062b50 con 0x7f60d8102a80 2024-04-21T10:29:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.250+0000 7f60d57fa640 1 -- 172.21.15.26:0/2354529515 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22220 (secure 0 0 0) 0x7f60c809c840 con 0x7f60d8102a80 2024-04-21T10:29:02.249 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:29:02.250 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":86,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:29:01.278953+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:29:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.252+0000 7f60de50d640 1 -- 172.21.15.26:0/2354529515 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f60c00a3bb0 msgr2=0x7f60c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.252+0000 7f60de50d640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f60c00a3bb0 0x7f60c00a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f60cc036040 tx=0x7f60cc000bc0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.252+0000 7f60de50d640 1 -- 172.21.15.26:0/2354529515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d8102a80 msgr2=0x7f60d8198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.252+0000 7f60de50d640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d8102a80 0x7f60d8198210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f60c800d950 tx=0x7f60c800de10 comp rx=0 tx=0).stop 2024-04-21T10:29:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.252+0000 7f60de50d640 1 -- 172.21.15.26:0/2354529515 shutdown_connections 2024-04-21T10:29:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.252+0000 7f60de50d640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f60c00a3bb0 0x7f60c00a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.252+0000 7f60de50d640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60d8107ce0 0x7f60d819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.252+0000 7f60de50d640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60d81053b0 0x7f60d8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.252+0000 7f60de50d640 1 --2- 172.21.15.26:0/2354529515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60d8102a80 0x7f60d8198210 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.252+0000 7f60de50d640 1 -- 172.21.15.26:0/2354529515 >> 172.21.15.26:0/2354529515 conn(0x7f60d80fc410 msgr2=0x7f60d81033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.252+0000 7f60de50d640 1 -- 172.21.15.26:0/2354529515 shutdown_connections 2024-04-21T10:29:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.252+0000 7f60de50d640 1 -- 172.21.15.26:0/2354529515 wait complete. 2024-04-21T10:29:02.263 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:29:02.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:01 smithi190 ceph-mon[31423]: pgmap v283: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:29:02.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:01 smithi190 ceph-mon[31423]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T10:29:02.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:01 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1720425560' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T10:29:02.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:01 smithi190 ceph-mon[31423]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T10:29:02.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:01 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2379170487' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T10:29:02.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:01 smithi190 ceph-mon[31423]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-21T10:29:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 -- 172.21.15.26:0/200596371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc78c103190 msgr2=0x7fc78c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 --2- 172.21.15.26:0/200596371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc78c103190 0x7fc78c103610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009900 tx=0x7fc77c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 -- 172.21.15.26:0/200596371 shutdown_connections 2024-04-21T10:29:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 --2- 172.21.15.26:0/200596371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc78c103b50 0x7fc78c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 --2- 172.21.15.26:0/200596371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc78c103190 0x7fc78c103610 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 --2- 172.21.15.26:0/200596371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc78c101fa0 0x7fc78c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 -- 172.21.15.26:0/200596371 >> 172.21.15.26:0/200596371 conn(0x7fc78c0fd530 msgr2=0x7fc78c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 -- 172.21.15.26:0/200596371 shutdown_connections 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 -- 172.21.15.26:0/200596371 wait complete. 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 Processor -- start 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 -- start start 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc78c101fa0 0x7fc78c198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.365+0000 7fc794506640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc78c103190 0x7fc78c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc794506640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc78c103b50 0x7fc78c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc794506640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c10db20 con 0x7fc78c103190 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc794506640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c10d9a0 con 0x7fc78c101fa0 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc794506640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c10dca0 con 0x7fc78c103b50 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc791aa1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc78c103190 0x7fc78c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc791aa1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc78c103190 0x7fc78c198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58698/0 (socket says 172.21.15.26:58698) 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc791aa1640 1 -- 172.21.15.26:0/4213652642 learned_addr learned my addr 172.21.15.26:0/4213652642 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc7922a2640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc78c101fa0 0x7fc78c198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc792aa3640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc78c103b50 0x7fc78c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc791aa1640 1 -- 172.21.15.26:0/4213652642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc78c103b50 msgr2=0x7fc78c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc791aa1640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc78c103b50 0x7fc78c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc791aa1640 1 -- 172.21.15.26:0/4213652642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc78c101fa0 msgr2=0x7fc78c198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc791aa1640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc78c101fa0 0x7fc78c198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc791aa1640 1 -- 172.21.15.26:0/4213652642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c19fd60 con 0x7fc78c103190 2024-04-21T10:29:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc7922a2640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc78c101fa0 0x7fc78c198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:29:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc791aa1640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc78c103190 0x7fc78c198680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc77c036040 tx=0x7fc77c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc7837fe640 1 -- 172.21.15.26:0/4213652642 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c043020 con 0x7fc78c103190 2024-04-21T10:29:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc7837fe640 1 -- 172.21.15.26:0/4213652642 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc77c0054c0 con 0x7fc78c103190 2024-04-21T10:29:02.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc7837fe640 1 -- 172.21.15.26:0/4213652642 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c003db0 con 0x7fc78c103190 2024-04-21T10:29:02.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc794506640 1 -- 172.21.15.26:0/4213652642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c19fff0 con 0x7fc78c103190 2024-04-21T10:29:02.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.366+0000 7fc794506640 1 -- 172.21.15.26:0/4213652642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c1a0530 con 0x7fc78c103190 2024-04-21T10:29:02.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.367+0000 7fc794506640 1 -- 172.21.15.26:0/4213652642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc78c109260 con 0x7fc78c103190 2024-04-21T10:29:02.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.368+0000 7fc7837fe640 1 -- 172.21.15.26:0/4213652642 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc77c046020 con 0x7fc78c103190 2024-04-21T10:29:02.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.369+0000 7fc7837fe640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc7780a3bb0 0x7fc7780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:02.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.369+0000 7fc7837fe640 1 -- 172.21.15.26:0/4213652642 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc77c0f2b70 con 0x7fc78c103190 2024-04-21T10:29:02.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.369+0000 7fc7922a2640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc7780a3bb0 0x7fc7780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:02.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.369+0000 7fc7922a2640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc7780a3bb0 0x7fc7780a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc78c102ff0 tx=0x7fc78800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:02.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.373+0000 7fc7837fe640 1 -- 172.21.15.26:0/4213652642 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc77c04d020 con 0x7fc78c103190 2024-04-21T10:29:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.561+0000 7fc794506640 1 -- 172.21.15.26:0/4213652642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc78c02d850 con 0x7fc78c103190 2024-04-21T10:29:02.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.563+0000 7fc7837fe640 1 -- 172.21.15.26:0/4213652642 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22220 (secure 0 0 0) 0x7fc77c0bf6a0 con 0x7fc78c103190 2024-04-21T10:29:02.563 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:29:02.563 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":86,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:29:01.278953+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:29:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.565+0000 7fc794506640 1 -- 172.21.15.26:0/4213652642 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc7780a3bb0 msgr2=0x7fc7780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.565+0000 7fc794506640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc7780a3bb0 0x7fc7780a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc78c102ff0 tx=0x7fc78800a5e0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.565+0000 7fc794506640 1 -- 172.21.15.26:0/4213652642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc78c103190 msgr2=0x7fc78c198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.565+0000 7fc794506640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc78c103190 0x7fc78c198680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc77c036040 tx=0x7fc77c0042f0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.565+0000 7fc794506640 1 -- 172.21.15.26:0/4213652642 shutdown_connections 2024-04-21T10:29:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.565+0000 7fc794506640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc7780a3bb0 0x7fc7780a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.565+0000 7fc794506640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc78c103b50 0x7fc78c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.565+0000 7fc794506640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc78c103190 0x7fc78c198680 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.565+0000 7fc794506640 1 --2- 172.21.15.26:0/4213652642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc78c101fa0 0x7fc78c198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.565+0000 7fc794506640 1 -- 172.21.15.26:0/4213652642 >> 172.21.15.26:0/4213652642 conn(0x7fc78c0fd530 msgr2=0x7fc78c106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:02.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.565+0000 7fc794506640 1 -- 172.21.15.26:0/4213652642 shutdown_connections 2024-04-21T10:29:02.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.565+0000 7fc794506640 1 -- 172.21.15.26:0/4213652642 wait complete. 2024-04-21T10:29:02.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-21T10:29:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.681+0000 7fd5d41b2640 1 -- 172.21.15.26:0/3432145271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5cc101160 msgr2=0x7fd5cc1015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.681+0000 7fd5d41b2640 1 --2- 172.21.15.26:0/3432145271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5cc101160 0x7fd5cc1015c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009900 tx=0x7fd5bc02bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.681+0000 7fd5d41b2640 1 -- 172.21.15.26:0/3432145271 shutdown_connections 2024-04-21T10:29:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.681+0000 7fd5d41b2640 1 --2- 172.21.15.26:0/3432145271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5cc101b00 0x7fd5cc101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.681+0000 7fd5d41b2640 1 --2- 172.21.15.26:0/3432145271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5cc101160 0x7fd5cc1015c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.681+0000 7fd5d41b2640 1 --2- 172.21.15.26:0/3432145271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5cc100800 0x7fd5cc100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.681+0000 7fd5d41b2640 1 -- 172.21.15.26:0/3432145271 >> 172.21.15.26:0/3432145271 conn(0x7fd5cc0fc400 msgr2=0x7fd5cc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.681+0000 7fd5d41b2640 1 -- 172.21.15.26:0/3432145271 shutdown_connections 2024-04-21T10:29:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.681+0000 7fd5d41b2640 1 -- 172.21.15.26:0/3432145271 wait complete. 2024-04-21T10:29:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.681+0000 7fd5d41b2640 1 Processor -- start 2024-04-21T10:29:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.681+0000 7fd5d41b2640 1 -- start start 2024-04-21T10:29:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d41b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5cc100800 0x7fd5cc19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d41b2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5cc101160 0x7fd5cc19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d41b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5cc101b00 0x7fd5cc1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d41b2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc10fd00 con 0x7fd5cc100800 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d41b2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc10fb80 con 0x7fd5cc101160 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d41b2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc10fe80 con 0x7fd5cc101b00 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d1f4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5cc100800 0x7fd5cc19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d1f4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5cc100800 0x7fd5cc19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58724/0 (socket says 172.21.15.26:58724) 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d1f4e640 1 -- 172.21.15.26:0/2362744188 learned_addr learned my addr 172.21.15.26:0/2362744188 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d274f640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5cc101b00 0x7fd5cc1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d1f4e640 1 -- 172.21.15.26:0/2362744188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5cc101b00 msgr2=0x7fd5cc1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d1f4e640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5cc101b00 0x7fd5cc1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d1f4e640 1 -- 172.21.15.26:0/2362744188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5cc101160 msgr2=0x7fd5cc19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d1f4e640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5cc101160 0x7fd5cc19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d1f4e640 1 -- 172.21.15.26:0/2362744188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc1a1fd0 con 0x7fd5cc100800 2024-04-21T10:29:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d1f4e640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5cc100800 0x7fd5cc19a3b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800cce0 tx=0x7fd5c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5c2ffd640 1 -- 172.21.15.26:0/2362744188 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800f650 con 0x7fd5cc100800 2024-04-21T10:29:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5c2ffd640 1 -- 172.21.15.26:0/2362744188 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c800fca0 con 0x7fd5cc100800 2024-04-21T10:29:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d41b2640 1 -- 172.21.15.26:0/2362744188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5cc1a22c0 con 0x7fd5cc100800 2024-04-21T10:29:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5c2ffd640 1 -- 172.21.15.26:0/2362744188 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8018b40 con 0x7fd5cc100800 2024-04-21T10:29:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.682+0000 7fd5d41b2640 1 -- 172.21.15.26:0/2362744188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5cc1a2810 con 0x7fd5cc100800 2024-04-21T10:29:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.683+0000 7fd5d41b2640 1 -- 172.21.15.26:0/2362744188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5cc10b740 con 0x7fd5cc100800 2024-04-21T10:29:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.684+0000 7fd5c2ffd640 1 -- 172.21.15.26:0/2362744188 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd5c800f7f0 con 0x7fd5cc100800 2024-04-21T10:29:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.685+0000 7fd5c2ffd640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd5b80a3bb0 0x7fd5b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.685+0000 7fd5d174d640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd5b80a3bb0 0x7fd5b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.685+0000 7fd5c2ffd640 1 -- 172.21.15.26:0/2362744188 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd5c80d08d0 con 0x7fd5cc100800 2024-04-21T10:29:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.686+0000 7fd5d174d640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd5b80a3bb0 0x7fd5b80a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc005c20 tx=0x7fd5bc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:02.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.690+0000 7fd5c2ffd640 1 -- 172.21.15.26:0/2362744188 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5c8014030 con 0x7fd5cc100800 2024-04-21T10:29:02.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.916+0000 7fd5d41b2640 1 -- 172.21.15.26:0/2362744188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd5cc062b50 con 0x7fd5cc100800 2024-04-21T10:29:02.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.925+0000 7fd5c2ffd640 1 -- 172.21.15.26:0/2362744188 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fd5c80bbe20 con 0x7fd5cc100800 2024-04-21T10:29:02.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.927+0000 7fd5d41b2640 1 -- 172.21.15.26:0/2362744188 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd5b80a3bb0 msgr2=0x7fd5b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.927+0000 7fd5d41b2640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd5b80a3bb0 0x7fd5b80a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc005c20 tx=0x7fd5bc000bc0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.927+0000 7fd5d41b2640 1 -- 172.21.15.26:0/2362744188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5cc100800 msgr2=0x7fd5cc19a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.927+0000 7fd5d41b2640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5cc100800 0x7fd5cc19a3b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800cce0 tx=0x7fd5c800a620 comp rx=0 tx=0).stop 2024-04-21T10:29:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.927+0000 7fd5d41b2640 1 -- 172.21.15.26:0/2362744188 shutdown_connections 2024-04-21T10:29:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.927+0000 7fd5d41b2640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd5b80a3bb0 0x7fd5b80a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.927+0000 7fd5d41b2640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5cc101b00 0x7fd5cc1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.927+0000 7fd5d41b2640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5cc101160 0x7fd5cc19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.927+0000 7fd5d41b2640 1 --2- 172.21.15.26:0/2362744188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5cc100800 0x7fd5cc19a3b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.927+0000 7fd5d41b2640 1 -- 172.21.15.26:0/2362744188 >> 172.21.15.26:0/2362744188 conn(0x7fd5cc0fc400 msgr2=0x7fd5cc0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.927+0000 7fd5d41b2640 1 -- 172.21.15.26:0/2362744188 shutdown_connections 2024-04-21T10:29:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:02.927+0000 7fd5d41b2640 1 -- 172.21.15.26:0/2362744188 wait complete. 2024-04-21T10:29:02.939 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-21T10:29:02.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-21T10:29:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.044+0000 7fe5d42c0640 1 -- 172.21.15.26:0/3979483164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5cc103b60 msgr2=0x7fe5cc10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.044+0000 7fe5d42c0640 1 --2- 172.21.15.26:0/3979483164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5cc103b60 0x7fe5cc10a3f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009ca0 tx=0x7fe5c402c0f0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 -- 172.21.15.26:0/3979483164 shutdown_connections 2024-04-21T10:29:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 --2- 172.21.15.26:0/3979483164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5cc103b60 0x7fe5cc10a3f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 --2- 172.21.15.26:0/3979483164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5cc1031a0 0x7fe5cc103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 --2- 172.21.15.26:0/3979483164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5cc101fb0 0x7fe5cc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 -- 172.21.15.26:0/3979483164 >> 172.21.15.26:0/3979483164 conn(0x7fe5cc0fd520 msgr2=0x7fe5cc0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 -- 172.21.15.26:0/3979483164 shutdown_connections 2024-04-21T10:29:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 -- 172.21.15.26:0/3979483164 wait complete. 2024-04-21T10:29:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 Processor -- start 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 -- start start 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5cc101fb0 0x7fe5cc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5cc1031a0 0x7fe5cc1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5cc103b60 0x7fe5cc19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10dbd0 con 0x7fe5cc101fb0 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10da50 con 0x7fe5cc103b60 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.045+0000 7fe5d42c0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10dd50 con 0x7fe5cc1031a0 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d205c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5cc101fb0 0x7fe5cc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d205c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5cc101fb0 0x7fe5cc198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58738/0 (socket says 172.21.15.26:58738) 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d205c640 1 -- 172.21.15.26:0/604535896 learned_addr learned my addr 172.21.15.26:0/604535896 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d285d640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5cc103b60 0x7fe5cc19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d185b640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5cc1031a0 0x7fe5cc1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d205c640 1 -- 172.21.15.26:0/604535896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5cc1031a0 msgr2=0x7fe5cc1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d205c640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5cc1031a0 0x7fe5cc1987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d205c640 1 -- 172.21.15.26:0/604535896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5cc103b60 msgr2=0x7fe5cc19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d205c640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5cc103b60 0x7fe5cc19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d205c640 1 -- 172.21.15.26:0/604535896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc19feb0 con 0x7fe5cc101fb0 2024-04-21T10:29:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d285d640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5cc103b60 0x7fe5cc19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:29:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d205c640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5cc101fb0 0x7fe5cc198290 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800b450 tx=0x7fe5c800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5c37fe640 1 -- 172.21.15.26:0/604535896 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8015020 con 0x7fe5cc101fb0 2024-04-21T10:29:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5c37fe640 1 -- 172.21.15.26:0/604535896 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5c8011d40 con 0x7fe5cc101fb0 2024-04-21T10:29:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d42c0640 1 -- 172.21.15.26:0/604535896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5cc1a01a0 con 0x7fe5cc101fb0 2024-04-21T10:29:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5c37fe640 1 -- 172.21.15.26:0/604535896 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c800f340 con 0x7fe5cc101fb0 2024-04-21T10:29:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.046+0000 7fe5d42c0640 1 -- 172.21.15.26:0/604535896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5cc1a07c0 con 0x7fe5cc101fb0 2024-04-21T10:29:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.047+0000 7fe5d42c0640 1 -- 172.21.15.26:0/604535896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5cc1093a0 con 0x7fe5cc101fb0 2024-04-21T10:29:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.048+0000 7fe5c37fe640 1 -- 172.21.15.26:0/604535896 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe5c8018020 con 0x7fe5cc101fb0 2024-04-21T10:29:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.049+0000 7fe5c37fe640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe5b80a3c30 0x7fe5b80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.049+0000 7fe5c37fe640 1 -- 172.21.15.26:0/604535896 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fe5c80cfe10 con 0x7fe5cc101fb0 2024-04-21T10:29:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.049+0000 7fe5d185b640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe5b80a3c30 0x7fe5b80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:03.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.050+0000 7fe5d185b640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe5b80a3c30 0x7fe5b80a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc1997a0 tx=0x7fe5bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:03.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.053+0000 7fe5c37fe640 1 -- 172.21.15.26:0/604535896 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5c80d3030 con 0x7fe5cc101fb0 2024-04-21T10:29:03.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:02 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2379170487' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T10:29:03.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:02 smithi026 ceph-mon[27965]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T10:29:03.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:02 smithi026 ceph-mon[27965]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-21T10:29:03.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:02 smithi026 ceph-mon[27965]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-21T10:29:03.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:02 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2354529515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:03.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:02 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/4213652642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:03.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:02 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2362744188' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T10:29:03.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:02 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2379170487' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T10:29:03.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:02 smithi169 ceph-mon[31469]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T10:29:03.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:02 smithi169 ceph-mon[31469]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-21T10:29:03.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:02 smithi169 ceph-mon[31469]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-21T10:29:03.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:02 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2354529515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:03.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:02 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/4213652642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:03.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:02 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2362744188' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T10:29:03.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:02 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2379170487' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T10:29:03.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:02 smithi190 ceph-mon[31423]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T10:29:03.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:02 smithi190 ceph-mon[31423]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-21T10:29:03.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:02 smithi190 ceph-mon[31423]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-21T10:29:03.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:02 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2354529515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:03.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:02 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/4213652642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:03.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:02 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2362744188' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T10:29:03.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.281+0000 7fe5d42c0640 1 -- 172.21.15.26:0/604535896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fe5cc062b50 con 0x7fe5cc101fb0 2024-04-21T10:29:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.793+0000 7fe5c37fe640 1 -- 172.21.15.26:0/604535896 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7fe5c801f020 con 0x7fe5cc101fb0 2024-04-21T10:29:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.796+0000 7fe5d42c0640 1 -- 172.21.15.26:0/604535896 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe5b80a3c30 msgr2=0x7fe5b80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.796+0000 7fe5d42c0640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe5b80a3c30 0x7fe5b80a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc1997a0 tx=0x7fe5bc008040 comp rx=0 tx=0).stop 2024-04-21T10:29:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.796+0000 7fe5d42c0640 1 -- 172.21.15.26:0/604535896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5cc101fb0 msgr2=0x7fe5cc198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.796+0000 7fe5d42c0640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5cc101fb0 0x7fe5cc198290 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800b450 tx=0x7fe5c800b910 comp rx=0 tx=0).stop 2024-04-21T10:29:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.796+0000 7fe5d42c0640 1 -- 172.21.15.26:0/604535896 shutdown_connections 2024-04-21T10:29:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.796+0000 7fe5d42c0640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe5b80a3c30 0x7fe5b80a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.796+0000 7fe5d42c0640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5cc103b60 0x7fe5cc19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.796+0000 7fe5d42c0640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe5cc1031a0 0x7fe5cc1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.796+0000 7fe5d42c0640 1 --2- 172.21.15.26:0/604535896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5cc101fb0 0x7fe5cc198290 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.796+0000 7fe5d42c0640 1 -- 172.21.15.26:0/604535896 >> 172.21.15.26:0/604535896 conn(0x7fe5cc0fd520 msgr2=0x7fe5cc106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.796+0000 7fe5d42c0640 1 -- 172.21.15.26:0/604535896 shutdown_connections 2024-04-21T10:29:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.796+0000 7fe5d42c0640 1 -- 172.21.15.26:0/604535896 wait complete. 2024-04-21T10:29:03.811 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-21T10:29:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.919+0000 7f88b7df4640 1 -- 172.21.15.26:0/3772162930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88b0100870 msgr2=0x7f88b0100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.919+0000 7f88b7df4640 1 --2- 172.21.15.26:0/3772162930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88b0100870 0x7f88b0100c90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009900 tx=0x7f88ac02bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 -- 172.21.15.26:0/3772162930 shutdown_connections 2024-04-21T10:29:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 --2- 172.21.15.26:0/3772162930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b0101b70 0x7f88b0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 --2- 172.21.15.26:0/3772162930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b01011d0 0x7f88b0101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 --2- 172.21.15.26:0/3772162930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88b0100870 0x7f88b0100c90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 -- 172.21.15.26:0/3772162930 >> 172.21.15.26:0/3772162930 conn(0x7f88b00fc410 msgr2=0x7f88b00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 -- 172.21.15.26:0/3772162930 shutdown_connections 2024-04-21T10:29:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 -- 172.21.15.26:0/3772162930 wait complete. 2024-04-21T10:29:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 Processor -- start 2024-04-21T10:29:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 -- start start 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b0100870 0x7f88b019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88b01011d0 0x7f88b019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b0101b70 0x7f88b01a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f88b010fcf0 con 0x7f88b01011d0 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88b010fb70 con 0x7f88b0100870 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.920+0000 7f88b7df4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f88b010fe70 con 0x7f88b0101b70 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b538f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88b01011d0 0x7f88b019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b538f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88b01011d0 0x7f88b019a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58750/0 (socket says 172.21.15.26:58750) 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b538f640 1 -- 172.21.15.26:0/2206639753 learned_addr learned my addr 172.21.15.26:0/2206639753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b6391640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b0101b70 0x7f88b01a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b5b90640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b0100870 0x7f88b019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b538f640 1 -- 172.21.15.26:0/2206639753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b0101b70 msgr2=0x7f88b01a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b538f640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b0101b70 0x7f88b01a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b538f640 1 -- 172.21.15.26:0/2206639753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b0100870 msgr2=0x7f88b019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:03.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b538f640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b0100870 0x7f88b019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:03.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b538f640 1 -- 172.21.15.26:0/2206639753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b01a2030 con 0x7f88b01011d0 2024-04-21T10:29:03.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b5b90640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b0100870 0x7f88b019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:29:03.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b538f640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88b01011d0 0x7f88b019a950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f88a000e9f0 tx=0x7f88a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:03.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88a6ffd640 1 -- 172.21.15.26:0/2206639753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a000cd80 con 0x7f88b01011d0 2024-04-21T10:29:03.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88a6ffd640 1 -- 172.21.15.26:0/2206639753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a00054d0 con 0x7f88b01011d0 2024-04-21T10:29:03.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88a6ffd640 1 -- 172.21.15.26:0/2206639753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0010690 con 0x7f88b01011d0 2024-04-21T10:29:03.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b7df4640 1 -- 172.21.15.26:0/2206639753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b01a2320 con 0x7f88b01011d0 2024-04-21T10:29:03.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.921+0000 7f88b7df4640 1 -- 172.21.15.26:0/2206639753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b01a2820 con 0x7f88b01011d0 2024-04-21T10:29:03.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.922+0000 7f88b7df4640 1 -- 172.21.15.26:0/2206639753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b010b850 con 0x7f88b01011d0 2024-04-21T10:29:03.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.923+0000 7f88a6ffd640 1 -- 172.21.15.26:0/2206639753 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f88a0004e30 con 0x7f88b01011d0 2024-04-21T10:29:03.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.924+0000 7f88a6ffd640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f889c0a3bb0 0x7f889c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:03.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.924+0000 7f88a6ffd640 1 -- 172.21.15.26:0/2206639753 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f88a00d09f0 con 0x7f88b01011d0 2024-04-21T10:29:03.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.924+0000 7f88b5b90640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f889c0a3bb0 0x7f889c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:03.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.925+0000 7f88b5b90640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f889c0a3bb0 0x7f889c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0096c0 tx=0x7f88ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:03.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:03.928+0000 7f88a6ffd640 1 -- 172.21.15.26:0/2206639753 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88a0053ff0 con 0x7f88b01011d0 2024-04-21T10:29:03.930 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i[62749]: 2024-04-21T10:29:03.632+0000 7fe309fc0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-21T10:29:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.170+0000 7f88b7df4640 1 -- 172.21.15.26:0/2206639753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f88b002d5c0 con 0x7f88b01011d0 2024-04-21T10:29:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: pgmap v286: 97 pgs: 25 creating+peering, 41 unknown, 31 active+clean; 578 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-04-21T10:29:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T10:29:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: Cluster is now healthy 2024-04-21T10:29:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2362744188' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T10:29:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] up:active 2024-04-21T10:29:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-21T10:29:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T10:29:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/604535896' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-21T10:29:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T10:29:04.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T10:29:04.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/604535896' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-21T10:29:04.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-21T10:29:04.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:03 smithi026 ceph-mon[27965]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-21T10:29:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: pgmap v286: 97 pgs: 25 creating+peering, 41 unknown, 31 active+clean; 578 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-04-21T10:29:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T10:29:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: Cluster is now healthy 2024-04-21T10:29:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2362744188' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T10:29:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] up:active 2024-04-21T10:29:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-21T10:29:04.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T10:29:04.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/604535896' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-21T10:29:04.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T10:29:04.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T10:29:04.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/604535896' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-21T10:29:04.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-21T10:29:04.199 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:03 smithi169 ceph-mon[31469]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-21T10:29:04.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: pgmap v286: 97 pgs: 25 creating+peering, 41 unknown, 31 active+clean; 578 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-04-21T10:29:04.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T10:29:04.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: Cluster is now healthy 2024-04-21T10:29:04.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2362744188' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T10:29:04.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] up:active 2024-04-21T10:29:04.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-21T10:29:04.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T10:29:04.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/604535896' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-21T10:29:04.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T10:29:04.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T10:29:04.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/604535896' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-21T10:29:04.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-21T10:29:04.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:03 smithi190 ceph-mon[31423]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-21T10:29:04.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.795+0000 7f88a6ffd640 1 -- 172.21.15.26:0/2206639753 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f88a001f340 con 0x7f88b01011d0 2024-04-21T10:29:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.799+0000 7f88b7df4640 1 -- 172.21.15.26:0/2206639753 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f889c0a3bb0 msgr2=0x7f889c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.799+0000 7f88b7df4640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f889c0a3bb0 0x7f889c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0096c0 tx=0x7f88ac000bc0 comp rx=0 tx=0).stop 2024-04-21T10:29:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.799+0000 7f88b7df4640 1 -- 172.21.15.26:0/2206639753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88b01011d0 msgr2=0x7f88b019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.799+0000 7f88b7df4640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88b01011d0 0x7f88b019a950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f88a000e9f0 tx=0x7f88a000edf0 comp rx=0 tx=0).stop 2024-04-21T10:29:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.799+0000 7f88b7df4640 1 -- 172.21.15.26:0/2206639753 shutdown_connections 2024-04-21T10:29:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.799+0000 7f88b7df4640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f889c0a3bb0 0x7f889c0a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.799+0000 7f88b7df4640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88b0101b70 0x7f88b01a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:04.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.799+0000 7f88b7df4640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88b01011d0 0x7f88b019a950 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:04.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.799+0000 7f88b7df4640 1 --2- 172.21.15.26:0/2206639753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88b0100870 0x7f88b019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:04.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.799+0000 7f88b7df4640 1 -- 172.21.15.26:0/2206639753 >> 172.21.15.26:0/2206639753 conn(0x7f88b00fc410 msgr2=0x7f88b00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:04.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.799+0000 7f88b7df4640 1 -- 172.21.15.26:0/2206639753 shutdown_connections 2024-04-21T10:29:04.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.799+0000 7f88b7df4640 1 -- 172.21.15.26:0/2206639753 wait complete. 2024-04-21T10:29:04.817 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-21T10:29:04.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.929+0000 7f4899855640 1 -- 172.21.15.26:0/1289342135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894102a30 msgr2=0x7f4894104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.929+0000 7f4899855640 1 --2- 172.21.15.26:0/1289342135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894102a30 0x7f4894104e20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4884009900 tx=0x7f488402bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.929+0000 7f4899855640 1 -- 172.21.15.26:0/1289342135 shutdown_connections 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.929+0000 7f4899855640 1 --2- 172.21.15.26:0/1289342135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4894107c90 0x7f489410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.929+0000 7f4899855640 1 --2- 172.21.15.26:0/1289342135 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4894105360 0x7f4894107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.929+0000 7f4899855640 1 --2- 172.21.15.26:0/1289342135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894102a30 0x7f4894104e20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.929+0000 7f4899855640 1 -- 172.21.15.26:0/1289342135 >> 172.21.15.26:0/1289342135 conn(0x7f48940fc440 msgr2=0x7f48940fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.929+0000 7f4899855640 1 -- 172.21.15.26:0/1289342135 shutdown_connections 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.929+0000 7f4899855640 1 -- 172.21.15.26:0/1289342135 wait complete. 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.930+0000 7f4899855640 1 Processor -- start 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.930+0000 7f4899855640 1 -- start start 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.930+0000 7f4899855640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4894102a30 0x7f48941023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.930+0000 7f4899855640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4894105360 0x7f48941009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.930+0000 7f4899855640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894107c90 0x7f4894100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.930+0000 7f4899855640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f489410dba0 con 0x7f4894107c90 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.930+0000 7f4899855640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f489410da20 con 0x7f4894105360 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.930+0000 7f4899855640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f489410dd20 con 0x7f4894102a30 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.930+0000 7f48937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894107c90 0x7f4894100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.930+0000 7f4892ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4894102a30 0x7f48941023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.930+0000 7f48937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894107c90 0x7f4894100f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58754/0 (socket says 172.21.15.26:58754) 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f48937fe640 1 -- 172.21.15.26:0/1198898071 learned_addr learned my addr 172.21.15.26:0/1198898071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.930+0000 7f4892ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4894102a30 0x7f48941023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.26:51336/0 (socket says 172.21.15.26:51336) 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f48927fc640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4894105360 0x7f48941009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f48937fe640 1 -- 172.21.15.26:0/1198898071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4894102a30 msgr2=0x7f48941023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f48937fe640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4894102a30 0x7f48941023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f48937fe640 1 -- 172.21.15.26:0/1198898071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4894105360 msgr2=0x7f48941009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f48937fe640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4894105360 0x7f48941009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f48937fe640 1 -- 172.21.15.26:0/1198898071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48941017b0 con 0x7f4894107c90 2024-04-21T10:29:04.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f4892ffd640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4894102a30 0x7f48941023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:29:04.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f48937fe640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894107c90 0x7f4894100f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f488c00eb10 tx=0x7f488c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:04.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f4898853640 1 -- 172.21.15.26:0/1198898071 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c00ce20 con 0x7f4894107c90 2024-04-21T10:29:04.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f4899855640 1 -- 172.21.15.26:0/1198898071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48941a23c0 con 0x7f4894107c90 2024-04-21T10:29:04.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f4898853640 1 -- 172.21.15.26:0/1198898071 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488c005020 con 0x7f4894107c90 2024-04-21T10:29:04.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f4898853640 1 -- 172.21.15.26:0/1198898071 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c010690 con 0x7f4894107c90 2024-04-21T10:29:04.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.931+0000 7f4899855640 1 -- 172.21.15.26:0/1198898071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48941a2990 con 0x7f4894107c90 2024-04-21T10:29:04.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.932+0000 7f4899855640 1 -- 172.21.15.26:0/1198898071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48940fdf30 con 0x7f4894107c90 2024-04-21T10:29:04.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.936+0000 7f4898853640 1 -- 172.21.15.26:0/1198898071 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f488c005510 con 0x7f4894107c90 2024-04-21T10:29:04.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.936+0000 7f4898853640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f487c0a3bb0 0x7f487c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:04.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.937+0000 7f4898853640 1 -- 172.21.15.26:0/1198898071 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f488c0d0a80 con 0x7f4894107c90 2024-04-21T10:29:04.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.937+0000 7f4892ffd640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f487c0a3bb0 0x7f487c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:04.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.937+0000 7f4892ffd640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f487c0a3bb0 0x7f487c0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4884000c00 tx=0x7f488402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:04.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:04.939+0000 7f4898853640 1 -- 172.21.15.26:0/1198898071 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f488c053ff0 con 0x7f4894107c90 2024-04-21T10:29:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.168+0000 7f4899855640 1 -- 172.21.15.26:0/1198898071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f489402d5c0 con 0x7f4894107c90 2024-04-21T10:29:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:04 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2206639753' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T10:29:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:04 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2206639753' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T10:29:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:04 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] up:active 2024-04-21T10:29:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:04 smithi026 ceph-mon[27965]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T10:29:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:04 smithi026 ceph-mon[27965]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T10:29:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:04 smithi026 ceph-mon[27965]: Cluster is now healthy 2024-04-21T10:29:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:04 smithi026 ceph-mon[27965]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-21T10:29:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:04 smithi026 ceph-mon[27965]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-21T10:29:05.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:04 smithi026 ceph-mon[27965]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-21T10:29:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:04 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2206639753' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T10:29:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:04 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2206639753' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T10:29:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:04 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] up:active 2024-04-21T10:29:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:04 smithi169 ceph-mon[31469]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T10:29:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:04 smithi169 ceph-mon[31469]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T10:29:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:04 smithi169 ceph-mon[31469]: Cluster is now healthy 2024-04-21T10:29:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:04 smithi169 ceph-mon[31469]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-21T10:29:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:04 smithi169 ceph-mon[31469]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-21T10:29:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:04 smithi169 ceph-mon[31469]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-21T10:29:05.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:04 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2206639753' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T10:29:05.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:04 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2206639753' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T10:29:05.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:04 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] up:active 2024-04-21T10:29:05.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:04 smithi190 ceph-mon[31423]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T10:29:05.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:04 smithi190 ceph-mon[31423]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T10:29:05.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:04 smithi190 ceph-mon[31423]: Cluster is now healthy 2024-04-21T10:29:05.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:04 smithi190 ceph-mon[31423]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-21T10:29:05.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:04 smithi190 ceph-mon[31423]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-21T10:29:05.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:04 smithi190 ceph-mon[31423]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-21T10:29:05.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.802+0000 7f4898853640 1 -- 172.21.15.26:0/1198898071 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v19) ==== 115+0+0 (secure 0 0 0) 0x7f488c09d5b0 con 0x7f4894107c90 2024-04-21T10:29:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.806+0000 7f4899855640 1 -- 172.21.15.26:0/1198898071 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f487c0a3bb0 msgr2=0x7f487c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.806+0000 7f4899855640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f487c0a3bb0 0x7f487c0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4884000c00 tx=0x7f488402c7e0 comp rx=0 tx=0).stop 2024-04-21T10:29:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.806+0000 7f4899855640 1 -- 172.21.15.26:0/1198898071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894107c90 msgr2=0x7f4894100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.806+0000 7f4899855640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894107c90 0x7f4894100f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f488c00eb10 tx=0x7f488c00ef10 comp rx=0 tx=0).stop 2024-04-21T10:29:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.806+0000 7f4899855640 1 -- 172.21.15.26:0/1198898071 shutdown_connections 2024-04-21T10:29:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.806+0000 7f4899855640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f487c0a3bb0 0x7f487c0a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.806+0000 7f4899855640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4894107c90 0x7f4894100f20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.806+0000 7f4899855640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4894105360 0x7f48941009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.806+0000 7f4899855640 1 --2- 172.21.15.26:0/1198898071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4894102a30 0x7f48941023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.806+0000 7f4899855640 1 -- 172.21.15.26:0/1198898071 >> 172.21.15.26:0/1198898071 conn(0x7f48940fc440 msgr2=0x7f4894103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.806+0000 7f4899855640 1 -- 172.21.15.26:0/1198898071 shutdown_connections 2024-04-21T10:29:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.806+0000 7f4899855640 1 -- 172.21.15.26:0/1198898071 wait complete. 2024-04-21T10:29:05.818 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-21T10:29:05.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-21T10:29:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.927+0000 7efcb9614640 1 -- 172.21.15.26:0/2290544617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcb4100e90 msgr2=0x7efcb41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.927+0000 7efcb9614640 1 --2- 172.21.15.26:0/2290544617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcb4100e90 0x7efcb41012b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009900 tx=0x7efc9c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.927+0000 7efcb9614640 1 -- 172.21.15.26:0/2290544617 shutdown_connections 2024-04-21T10:29:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.927+0000 7efcb9614640 1 --2- 172.21.15.26:0/2290544617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb41029a0 0x7efcb41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.927+0000 7efcb9614640 1 --2- 172.21.15.26:0/2290544617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb4101fe0 0x7efcb4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.927+0000 7efcb9614640 1 --2- 172.21.15.26:0/2290544617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcb4100e90 0x7efcb41012b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.927+0000 7efcb9614640 1 -- 172.21.15.26:0/2290544617 >> 172.21.15.26:0/2290544617 conn(0x7efcb40fc400 msgr2=0x7efcb40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.927+0000 7efcb9614640 1 -- 172.21.15.26:0/2290544617 shutdown_connections 2024-04-21T10:29:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.927+0000 7efcb9614640 1 -- 172.21.15.26:0/2290544617 wait complete. 2024-04-21T10:29:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.927+0000 7efcb9614640 1 Processor -- start 2024-04-21T10:29:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb9614640 1 -- start start 2024-04-21T10:29:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb9614640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb4100e90 0x7efcb4069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb9614640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb4101fe0 0x7efcb419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb9614640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcb41029a0 0x7efcb419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb9614640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efcb410c9c0 con 0x7efcb41029a0 2024-04-21T10:29:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb9614640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efcb410c840 con 0x7efcb4100e90 2024-04-21T10:29:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb9614640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efcb410cb40 con 0x7efcb4101fe0 2024-04-21T10:29:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcb41029a0 0x7efcb419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcb41029a0 0x7efcb419f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58770/0 (socket says 172.21.15.26:58770) 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb37fe640 1 -- 172.21.15.26:0/1711012008 learned_addr learned my addr 172.21.15.26:0/1711012008 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb2ffd640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb4100e90 0x7efcb4069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb27fc640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb4101fe0 0x7efcb419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb37fe640 1 -- 172.21.15.26:0/1711012008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb4101fe0 msgr2=0x7efcb419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb37fe640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb4101fe0 0x7efcb419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb37fe640 1 -- 172.21.15.26:0/1711012008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb4100e90 msgr2=0x7efcb4069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb37fe640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb4100e90 0x7efcb4069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb37fe640 1 -- 172.21.15.26:0/1711012008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb419fe20 con 0x7efcb41029a0 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcb37fe640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcb41029a0 0x7efcb419f560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efcac00eaa0 tx=0x7efcac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.928+0000 7efcabfff640 1 -- 172.21.15.26:0/1711012008 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac00cdb0 con 0x7efcb41029a0 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.929+0000 7efcabfff640 1 -- 172.21.15.26:0/1711012008 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcac005020 con 0x7efcb41029a0 2024-04-21T10:29:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.929+0000 7efcb9614640 1 -- 172.21.15.26:0/1711012008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb41a0110 con 0x7efcb41029a0 2024-04-21T10:29:05.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.929+0000 7efcabfff640 1 -- 172.21.15.26:0/1711012008 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac0116d0 con 0x7efcb41029a0 2024-04-21T10:29:05.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.929+0000 7efcb9614640 1 -- 172.21.15.26:0/1711012008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb41a0660 con 0x7efcb41029a0 2024-04-21T10:29:05.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.930+0000 7efcb9614640 1 -- 172.21.15.26:0/1711012008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb4108310 con 0x7efcb41029a0 2024-04-21T10:29:05.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.931+0000 7efcabfff640 1 -- 172.21.15.26:0/1711012008 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efcac0051d0 con 0x7efcb41029a0 2024-04-21T10:29:05.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.931+0000 7efcabfff640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7efca00a3c30 0x7efca00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:05.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.932+0000 7efcb2ffd640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7efca00a3c30 0x7efca00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:05.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.932+0000 7efcabfff640 1 -- 172.21.15.26:0/1711012008 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7efcac0d0390 con 0x7efcb41029a0 2024-04-21T10:29:05.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.932+0000 7efcb2ffd640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7efca00a3c30 0x7efca00a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009230 tx=0x7efc9c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:05.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:05.936+0000 7efcabfff640 1 -- 172.21.15.26:0/1711012008 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efcac016030 con 0x7efcb41029a0 2024-04-21T10:29:06.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.149+0000 7efcb9614640 1 -- 172.21.15.26:0/1711012008 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efcb4062390 con 0x7efca00a3c30 2024-04-21T10:29:06.155 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:05 smithi026 ceph-mon[27965]: pgmap v288: 97 pgs: 25 creating+peering, 12 unknown, 60 active+clean; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 3 op/s 2024-04-21T10:29:06.155 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:05 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1198898071' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T10:29:06.155 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:05 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1198898071' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T10:29:06.155 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:05 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] up:active 2024-04-21T10:29:06.155 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:05 smithi026 ceph-mon[27965]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-04-21T10:29:06.155 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:05 smithi026 ceph-mon[27965]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:29:06.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.173+0000 7efcabfff640 1 -- 172.21.15.26:0/1711012008 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efcb4062390 con 0x7efca00a3c30 2024-04-21T10:29:06.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.175+0000 7efcb9614640 1 -- 172.21.15.26:0/1711012008 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7efca00a3c30 msgr2=0x7efca00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:06.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.175+0000 7efcb9614640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7efca00a3c30 0x7efca00a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009230 tx=0x7efc9c005fd0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.175+0000 7efcb9614640 1 -- 172.21.15.26:0/1711012008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcb41029a0 msgr2=0x7efcb419f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:06.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.175+0000 7efcb9614640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcb41029a0 0x7efcb419f560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efcac00eaa0 tx=0x7efcac00ef60 comp rx=0 tx=0).stop 2024-04-21T10:29:06.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.175+0000 7efcb9614640 1 -- 172.21.15.26:0/1711012008 shutdown_connections 2024-04-21T10:29:06.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.175+0000 7efcb9614640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7efca00a3c30 0x7efca00a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.175+0000 7efcb9614640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcb41029a0 0x7efcb419f560 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.175+0000 7efcb9614640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb4101fe0 0x7efcb419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.175+0000 7efcb9614640 1 --2- 172.21.15.26:0/1711012008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb4100e90 0x7efcb4069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.175+0000 7efcb9614640 1 -- 172.21.15.26:0/1711012008 >> 172.21.15.26:0/1711012008 conn(0x7efcb40fc400 msgr2=0x7efcb41054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:06.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.175+0000 7efcb9614640 1 -- 172.21.15.26:0/1711012008 shutdown_connections 2024-04-21T10:29:06.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.175+0000 7efcb9614640 1 -- 172.21.15.26:0/1711012008 wait complete. 2024-04-21T10:29:06.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-21T10:29:06.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:05 smithi169 ceph-mon[31469]: pgmap v288: 97 pgs: 25 creating+peering, 12 unknown, 60 active+clean; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 3 op/s 2024-04-21T10:29:06.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:05 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1198898071' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T10:29:06.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:05 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1198898071' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T10:29:06.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:05 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] up:active 2024-04-21T10:29:06.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:05 smithi169 ceph-mon[31469]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-04-21T10:29:06.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:05 smithi169 ceph-mon[31469]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:29:06.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:05 smithi190 ceph-mon[31423]: pgmap v288: 97 pgs: 25 creating+peering, 12 unknown, 60 active+clean; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 3 op/s 2024-04-21T10:29:06.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:05 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1198898071' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T10:29:06.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:05 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1198898071' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T10:29:06.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:05 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] up:active 2024-04-21T10:29:06.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:05 smithi190 ceph-mon[31423]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-04-21T10:29:06.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:05 smithi190 ceph-mon[31423]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:29:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.296+0000 7effa8b85640 1 -- 172.21.15.26:0/3209797158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa4103190 msgr2=0x7effa4103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.296+0000 7effa8b85640 1 --2- 172.21.15.26:0/3209797158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa4103190 0x7effa4103610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7eff98009900 tx=0x7eff9802bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.296+0000 7effa8b85640 1 -- 172.21.15.26:0/3209797158 shutdown_connections 2024-04-21T10:29:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.296+0000 7effa8b85640 1 --2- 172.21.15.26:0/3209797158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effa4103b50 0x7effa410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.296+0000 7effa8b85640 1 --2- 172.21.15.26:0/3209797158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa4103190 0x7effa4103610 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.296+0000 7effa8b85640 1 --2- 172.21.15.26:0/3209797158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effa4101fa0 0x7effa41023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.296+0000 7effa8b85640 1 -- 172.21.15.26:0/3209797158 >> 172.21.15.26:0/3209797158 conn(0x7effa40fd530 msgr2=0x7effa40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.296+0000 7effa8b85640 1 -- 172.21.15.26:0/3209797158 shutdown_connections 2024-04-21T10:29:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.296+0000 7effa8b85640 1 -- 172.21.15.26:0/3209797158 wait complete. 2024-04-21T10:29:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa8b85640 1 Processor -- start 2024-04-21T10:29:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa8b85640 1 -- start start 2024-04-21T10:29:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa8b85640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa4101fa0 0x7effa41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa8b85640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effa4103190 0x7effa4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa8b85640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effa4103b50 0x7effa419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa8b85640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7effa410dae0 con 0x7effa4101fa0 2024-04-21T10:29:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa8b85640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7effa410d960 con 0x7effa4103190 2024-04-21T10:29:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa8b85640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7effa410dc60 con 0x7effa4103b50 2024-04-21T10:29:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa4101fa0 0x7effa41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa4101fa0 0x7effa41981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58822/0 (socket says 172.21.15.26:58822) 2024-04-21T10:29:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa259c640 1 -- 172.21.15.26:0/1040404318 learned_addr learned my addr 172.21.15.26:0/1040404318 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa2d9d640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effa4103b50 0x7effa419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.297+0000 7effa1d9b640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effa4103190 0x7effa4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.298+0000 7effa259c640 1 -- 172.21.15.26:0/1040404318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effa4103b50 msgr2=0x7effa419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.298+0000 7effa259c640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effa4103b50 0x7effa419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.298+0000 7effa259c640 1 -- 172.21.15.26:0/1040404318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effa4103190 msgr2=0x7effa4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.298+0000 7effa259c640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effa4103190 0x7effa4198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.298+0000 7effa259c640 1 -- 172.21.15.26:0/1040404318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa419fde0 con 0x7effa4101fa0 2024-04-21T10:29:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.298+0000 7effa259c640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa4101fa0 0x7effa41981c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00ea80 tx=0x7eff9c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.298+0000 7eff937fe640 1 -- 172.21.15.26:0/1040404318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00ce40 con 0x7effa4101fa0 2024-04-21T10:29:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.298+0000 7effa8b85640 1 -- 172.21.15.26:0/1040404318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa41a00d0 con 0x7effa4101fa0 2024-04-21T10:29:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.298+0000 7eff937fe640 1 -- 172.21.15.26:0/1040404318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff9c011920 con 0x7effa4101fa0 2024-04-21T10:29:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.298+0000 7eff937fe640 1 -- 172.21.15.26:0/1040404318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c01aaa0 con 0x7effa4101fa0 2024-04-21T10:29:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.298+0000 7effa8b85640 1 -- 172.21.15.26:0/1040404318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa41a05f0 con 0x7effa4101fa0 2024-04-21T10:29:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.299+0000 7effa8b85640 1 -- 172.21.15.26:0/1040404318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa4109360 con 0x7effa4101fa0 2024-04-21T10:29:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.300+0000 7eff937fe640 1 -- 172.21.15.26:0/1040404318 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7eff9c01ac40 con 0x7effa4101fa0 2024-04-21T10:29:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.301+0000 7eff937fe640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7eff8c0a3bb0 0x7eff8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:06.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.301+0000 7effa1d9b640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7eff8c0a3bb0 0x7eff8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:06.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.301+0000 7eff937fe640 1 -- 172.21.15.26:0/1040404318 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7eff9c0d07c0 con 0x7effa4101fa0 2024-04-21T10:29:06.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.301+0000 7effa1d9b640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7eff8c0a3bb0 0x7eff8c0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7eff98009230 tx=0x7eff98005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:06.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.305+0000 7eff937fe640 1 -- 172.21.15.26:0/1040404318 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff9c016030 con 0x7effa4101fa0 2024-04-21T10:29:06.435 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:29:06 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:29:06.153+0000 7f18f169e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:29:06.435 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:29:06 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:29:06.153+0000 7f18f169e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:29:06.436 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:29:06 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:29:06.153+0000 7f18f169e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:29:06.436 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:29:06 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:29:06.153+0000 7f18f169e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:29:06.436 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:29:06 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:29:06.153+0000 7f18f169e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:29:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.529+0000 7effa8b85640 1 -- 172.21.15.26:0/1040404318 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7effa4062390 con 0x7eff8c0a3bb0 2024-04-21T10:29:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.553+0000 7eff937fe640 1 -- 172.21.15.26:0/1040404318 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7effa4062390 con 0x7eff8c0a3bb0 2024-04-21T10:29:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.555+0000 7effa8b85640 1 -- 172.21.15.26:0/1040404318 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7eff8c0a3bb0 msgr2=0x7eff8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.555+0000 7effa8b85640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7eff8c0a3bb0 0x7eff8c0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7eff98009230 tx=0x7eff98005fd0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.555+0000 7effa8b85640 1 -- 172.21.15.26:0/1040404318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa4101fa0 msgr2=0x7effa41981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.555+0000 7effa8b85640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa4101fa0 0x7effa41981c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00ea80 tx=0x7eff9c00ef40 comp rx=0 tx=0).stop 2024-04-21T10:29:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.555+0000 7effa8b85640 1 -- 172.21.15.26:0/1040404318 shutdown_connections 2024-04-21T10:29:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.555+0000 7effa8b85640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7eff8c0a3bb0 0x7eff8c0a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.555+0000 7effa8b85640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effa4103b50 0x7effa419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.555+0000 7effa8b85640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effa4103190 0x7effa4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.555+0000 7effa8b85640 1 --2- 172.21.15.26:0/1040404318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effa4101fa0 0x7effa41981c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.555+0000 7effa8b85640 1 -- 172.21.15.26:0/1040404318 >> 172.21.15.26:0/1040404318 conn(0x7effa40fd530 msgr2=0x7effa4106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.555+0000 7effa8b85640 1 -- 172.21.15.26:0/1040404318 shutdown_connections 2024-04-21T10:29:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.555+0000 7effa8b85640 1 -- 172.21.15.26:0/1040404318 wait complete. 2024-04-21T10:29:06.566 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-21T10:29:06.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.714+0000 7f19ff38e640 1 -- 172.21.15.26:0/1012349687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19f8107c90 msgr2=0x7f19f810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:06.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.714+0000 7f19ff38e640 1 --2- 172.21.15.26:0/1012349687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19f8107c90 0x7f19f810a0e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009900 tx=0x7f19f002bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:06.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.714+0000 7f19ff38e640 1 -- 172.21.15.26:0/1012349687 shutdown_connections 2024-04-21T10:29:06.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.714+0000 7f19ff38e640 1 --2- 172.21.15.26:0/1012349687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19f8107c90 0x7f19f810a0e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.714+0000 7f19ff38e640 1 --2- 172.21.15.26:0/1012349687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f8105360 0x7f19f8107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.714+0000 7f19ff38e640 1 --2- 172.21.15.26:0/1012349687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19f8102a30 0x7f19f8104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.714+0000 7f19ff38e640 1 -- 172.21.15.26:0/1012349687 >> 172.21.15.26:0/1012349687 conn(0x7f19f80fc440 msgr2=0x7f19f80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.714+0000 7f19ff38e640 1 -- 172.21.15.26:0/1012349687 shutdown_connections 2024-04-21T10:29:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.714+0000 7f19ff38e640 1 -- 172.21.15.26:0/1012349687 wait complete. 2024-04-21T10:29:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19ff38e640 1 Processor -- start 2024-04-21T10:29:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19ff38e640 1 -- start start 2024-04-21T10:29:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19ff38e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19f8102a30 0x7f19f8198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19ff38e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f8105360 0x7f19f8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19ff38e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19f8107c90 0x7f19f819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19ff38e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f19f810daf0 con 0x7f19f8107c90 2024-04-21T10:29:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19ff38e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19f810d970 con 0x7f19f8105360 2024-04-21T10:29:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19ff38e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f19f810dc70 con 0x7f19f8102a30 2024-04-21T10:29:06.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19fd92b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19f8107c90 0x7f19f819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19fd92b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19f8107c90 0x7f19f819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58848/0 (socket says 172.21.15.26:58848) 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19fd92b640 1 -- 172.21.15.26:0/1806466532 learned_addr learned my addr 172.21.15.26:0/1806466532 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19fd12a640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19f8102a30 0x7f19f8198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19fd92b640 1 -- 172.21.15.26:0/1806466532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19f8102a30 msgr2=0x7f19f8198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19fd92b640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19f8102a30 0x7f19f8198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19fd92b640 1 -- 172.21.15.26:0/1806466532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f8105360 msgr2=0x7f19f8198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19fd92b640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f8105360 0x7f19f8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19fd92b640 1 -- 172.21.15.26:0/1806466532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f819fd70 con 0x7f19f8107c90 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.715+0000 7f19fd92b640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19f8107c90 0x7f19f819f700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009230 tx=0x7f19f0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.716+0000 7f19f67fc640 1 -- 172.21.15.26:0/1806466532 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0043020 con 0x7f19f8107c90 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.716+0000 7f19f67fc640 1 -- 172.21.15.26:0/1806466532 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19f00043d0 con 0x7f19f8107c90 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.716+0000 7f19f67fc640 1 -- 172.21.15.26:0/1806466532 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0003af0 con 0x7f19f8107c90 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.716+0000 7f19ff38e640 1 -- 172.21.15.26:0/1806466532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f81a0000 con 0x7f19f8107c90 2024-04-21T10:29:06.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.716+0000 7f19ff38e640 1 -- 172.21.15.26:0/1806466532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f81a0550 con 0x7f19f8107c90 2024-04-21T10:29:06.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.717+0000 7f19ff38e640 1 -- 172.21.15.26:0/1806466532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f80fdf30 con 0x7f19f8107c90 2024-04-21T10:29:06.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.718+0000 7f19f67fc640 1 -- 172.21.15.26:0/1806466532 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f19f0046020 con 0x7f19f8107c90 2024-04-21T10:29:06.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.719+0000 7f19f67fc640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f19e00a3bb0 0x7f19e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:06.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.719+0000 7f19fd12a640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f19e00a3bb0 0x7f19e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:06.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.719+0000 7f19f67fc640 1 -- 172.21.15.26:0/1806466532 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f19f00f2530 con 0x7f19f8107c90 2024-04-21T10:29:06.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.719+0000 7f19fd12a640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f19e00a3bb0 0x7f19e00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f19e8005fd0 tx=0x7f19e8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:06.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.723+0000 7f19f67fc640 1 -- 172.21.15.26:0/1806466532 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19f004d020 con 0x7f19f8107c90 2024-04-21T10:29:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.938+0000 7f19ff38e640 1 -- 172.21.15.26:0/1806466532 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f19f8062420 con 0x7f19e00a3bb0 2024-04-21T10:29:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.943+0000 7f19f67fc640 1 -- 172.21.15.26:0/1806466532 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f19f8062420 con 0x7f19e00a3bb0 2024-04-21T10:29:06.943 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/qa/sv_0/59282e9c-9229-438b-86bd-477730b8bb78 2024-04-21T10:29:06.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.946+0000 7f19ff38e640 1 -- 172.21.15.26:0/1806466532 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f19e00a3bb0 msgr2=0x7f19e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.946+0000 7f19ff38e640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f19e00a3bb0 0x7f19e00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f19e8005fd0 tx=0x7f19e8009500 comp rx=0 tx=0).stop 2024-04-21T10:29:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.946+0000 7f19ff38e640 1 -- 172.21.15.26:0/1806466532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19f8107c90 msgr2=0x7f19f819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.946+0000 7f19ff38e640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19f8107c90 0x7f19f819f700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009230 tx=0x7f19f0005e40 comp rx=0 tx=0).stop 2024-04-21T10:29:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.946+0000 7f19ff38e640 1 -- 172.21.15.26:0/1806466532 shutdown_connections 2024-04-21T10:29:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.946+0000 7f19ff38e640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f19e00a3bb0 0x7f19e00a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.946+0000 7f19ff38e640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19f8107c90 0x7f19f819f700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.946+0000 7f19ff38e640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f8105360 0x7f19f8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.946+0000 7f19ff38e640 1 --2- 172.21.15.26:0/1806466532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19f8102a30 0x7f19f8198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.946+0000 7f19ff38e640 1 -- 172.21.15.26:0/1806466532 >> 172.21.15.26:0/1806466532 conn(0x7f19f80fc440 msgr2=0x7f19f8103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.946+0000 7f19ff38e640 1 -- 172.21.15.26:0/1806466532 shutdown_connections 2024-04-21T10:29:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:06.946+0000 7f19ff38e640 1 -- 172.21.15.26:0/1806466532 wait complete. 2024-04-21T10:29:06.960 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-21T10:29:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.067+0000 7f587bb9d640 1 -- 172.21.15.26:0/773132627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5874103230 msgr2=0x7f58741036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.067+0000 7f587bb9d640 1 --2- 172.21.15.26:0/773132627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5874103230 0x7f58741036b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5864009930 tx=0x7f586402bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.067+0000 7f587bb9d640 1 -- 172.21.15.26:0/773132627 shutdown_connections 2024-04-21T10:29:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.067+0000 7f587bb9d640 1 --2- 172.21.15.26:0/773132627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5874103bf0 0x7f587410a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.067+0000 7f587bb9d640 1 --2- 172.21.15.26:0/773132627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5874103230 0x7f58741036b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.067+0000 7f587bb9d640 1 --2- 172.21.15.26:0/773132627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5874100260 0x7f5874100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.067+0000 7f587bb9d640 1 -- 172.21.15.26:0/773132627 >> 172.21.15.26:0/773132627 conn(0x7f58740fbb90 msgr2=0x7f58740fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.067+0000 7f587bb9d640 1 -- 172.21.15.26:0/773132627 shutdown_connections 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.067+0000 7f587bb9d640 1 -- 172.21.15.26:0/773132627 wait complete. 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587bb9d640 1 Processor -- start 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587bb9d640 1 -- start start 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587bb9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5874100260 0x7f5874198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587bb9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5874103230 0x7f58741987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587bb9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5874103bf0 0x7f587419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587bb9d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f587410db60 con 0x7f5874103bf0 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587bb9d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f587410d9e0 con 0x7f5874103230 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587bb9d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f587410dce0 con 0x7f5874100260 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587a13a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5874103bf0 0x7f587419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587a13a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5874103bf0 0x7f587419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58868/0 (socket says 172.21.15.26:58868) 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587a13a640 1 -- 172.21.15.26:0/408441758 learned_addr learned my addr 172.21.15.26:0/408441758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f5879939640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5874100260 0x7f5874198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f5879138640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5874103230 0x7f58741987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587a13a640 1 -- 172.21.15.26:0/408441758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5874100260 msgr2=0x7f5874198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587a13a640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5874100260 0x7f5874198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587a13a640 1 -- 172.21.15.26:0/408441758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5874103230 msgr2=0x7f58741987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587a13a640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5874103230 0x7f58741987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.068+0000 7f587a13a640 1 -- 172.21.15.26:0/408441758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587419fe80 con 0x7f5874103bf0 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.069+0000 7f5879138640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5874103230 0x7f58741987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.069+0000 7f587a13a640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5874103bf0 0x7f587419f810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f586c00b970 tx=0x7f586c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.069+0000 7f586affd640 1 -- 172.21.15.26:0/408441758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c019070 con 0x7f5874103bf0 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.069+0000 7f587bb9d640 1 -- 172.21.15.26:0/408441758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58741a0170 con 0x7f5874103bf0 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.069+0000 7f586affd640 1 -- 172.21.15.26:0/408441758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586c014440 con 0x7f5874103bf0 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.069+0000 7f586affd640 1 -- 172.21.15.26:0/408441758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c0075d0 con 0x7f5874103bf0 2024-04-21T10:29:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.069+0000 7f587bb9d640 1 -- 172.21.15.26:0/408441758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58741a06f0 con 0x7f5874103bf0 2024-04-21T10:29:07.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.070+0000 7f587bb9d640 1 -- 172.21.15.26:0/408441758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58741093d0 con 0x7f5874103bf0 2024-04-21T10:29:07.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.071+0000 7f586affd640 1 -- 172.21.15.26:0/408441758 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f586c01c070 con 0x7f5874103bf0 2024-04-21T10:29:07.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.072+0000 7f586affd640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f58600a3c40 0x7f58600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:07.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.072+0000 7f5879939640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f58600a3c40 0x7f58600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:07.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.072+0000 7f586affd640 1 -- 172.21.15.26:0/408441758 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f586c0cf850 con 0x7f5874103bf0 2024-04-21T10:29:07.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.072+0000 7f5879939640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f58600a3c40 0x7f58600a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f587406a3d0 tx=0x7f5870005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.076+0000 7f586affd640 1 -- 172.21.15.26:0/408441758 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f586c09bd40 con 0x7f5874103bf0 2024-04-21T10:29:07.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.293+0000 7f587bb9d640 1 -- 172.21.15.26:0/408441758 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5874062390 con 0x7f58600a3c40 2024-04-21T10:29:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.317+0000 7f586affd640 1 -- 172.21.15.26:0/408441758 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5874062390 con 0x7f58600a3c40 2024-04-21T10:29:07.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.319+0000 7f587bb9d640 1 -- 172.21.15.26:0/408441758 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f58600a3c40 msgr2=0x7f58600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.319+0000 7f587bb9d640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f58600a3c40 0x7f58600a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f587406a3d0 tx=0x7f5870005f90 comp rx=0 tx=0).stop 2024-04-21T10:29:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.319+0000 7f587bb9d640 1 -- 172.21.15.26:0/408441758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5874103bf0 msgr2=0x7f587419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.319+0000 7f587bb9d640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5874103bf0 0x7f587419f810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f586c00b970 tx=0x7f586c00be30 comp rx=0 tx=0).stop 2024-04-21T10:29:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.320+0000 7f587bb9d640 1 -- 172.21.15.26:0/408441758 shutdown_connections 2024-04-21T10:29:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.320+0000 7f587bb9d640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f58600a3c40 0x7f58600a6100 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.320+0000 7f587bb9d640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5874103bf0 0x7f587419f810 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.320+0000 7f587bb9d640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5874103230 0x7f58741987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.320+0000 7f587bb9d640 1 --2- 172.21.15.26:0/408441758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5874100260 0x7f5874198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.320+0000 7f587bb9d640 1 -- 172.21.15.26:0/408441758 >> 172.21.15.26:0/408441758 conn(0x7f58740fbb90 msgr2=0x7f58741065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.320+0000 7f587bb9d640 1 -- 172.21.15.26:0/408441758 shutdown_connections 2024-04-21T10:29:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.320+0000 7f587bb9d640 1 -- 172.21.15.26:0/408441758 wait complete. 2024-04-21T10:29:07.330 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-21T10:29:07.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.437+0000 7f84c8232640 1 -- 172.21.15.26:0/3221335642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84c0101fc0 msgr2=0x7f84c0102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.437+0000 7f84c8232640 1 --2- 172.21.15.26:0/3221335642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84c0101fc0 0x7f84c0102440 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009900 tx=0x7f84b402bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:07.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.437+0000 7f84c8232640 1 -- 172.21.15.26:0/3221335642 shutdown_connections 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.437+0000 7f84c8232640 1 --2- 172.21.15.26:0/3221335642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c0102980 0x7f84c01092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.437+0000 7f84c8232640 1 --2- 172.21.15.26:0/3221335642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84c0101fc0 0x7f84c0102440 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.437+0000 7f84c8232640 1 --2- 172.21.15.26:0/3221335642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c0100e70 0x7f84c0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c8232640 1 -- 172.21.15.26:0/3221335642 >> 172.21.15.26:0/3221335642 conn(0x7f84c00fc3e0 msgr2=0x7f84c00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c8232640 1 -- 172.21.15.26:0/3221335642 shutdown_connections 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c8232640 1 -- 172.21.15.26:0/3221335642 wait complete. 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c8232640 1 Processor -- start 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c8232640 1 -- start start 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c8232640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c0100e70 0x7f84c0104df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c8232640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c0101fc0 0x7f84c0103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c8232640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84c0102980 0x7f84c0103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c8232640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f84c010c9c0 con 0x7f84c0102980 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c8232640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84c010c840 con 0x7f84c0100e70 2024-04-21T10:29:07.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c8232640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f84c010cb40 con 0x7f84c0101fc0 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c67cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84c0102980 0x7f84c0103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c67cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84c0102980 0x7f84c0103970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58886/0 (socket says 172.21.15.26:58886) 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.438+0000 7f84c67cf640 1 -- 172.21.15.26:0/869598269 learned_addr learned my addr 172.21.15.26:0/869598269 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84c57cd640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c0101fc0 0x7f84c0103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84c5fce640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c0100e70 0x7f84c0104df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84c67cf640 1 -- 172.21.15.26:0/869598269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c0101fc0 msgr2=0x7f84c0103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84c67cf640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c0101fc0 0x7f84c0103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84c67cf640 1 -- 172.21.15.26:0/869598269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c0100e70 msgr2=0x7f84c0104df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84c67cf640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c0100e70 0x7f84c0104df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84c67cf640 1 -- 172.21.15.26:0/869598269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c0104200 con 0x7f84c0102980 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84c67cf640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84c0102980 0x7f84c0103970 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f84b800eaa0 tx=0x7f84b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84b2ffd640 1 -- 172.21.15.26:0/869598269 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b800cdb0 con 0x7f84c0102980 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84b2ffd640 1 -- 172.21.15.26:0/869598269 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b8005020 con 0x7f84c0102980 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84b2ffd640 1 -- 172.21.15.26:0/869598269 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80116d0 con 0x7f84c0102980 2024-04-21T10:29:07.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84c8232640 1 -- 172.21.15.26:0/869598269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c01a2330 con 0x7f84c0102980 2024-04-21T10:29:07.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.439+0000 7f84c8232640 1 -- 172.21.15.26:0/869598269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c01a2800 con 0x7f84c0102980 2024-04-21T10:29:07.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.440+0000 7f84c8232640 1 -- 172.21.15.26:0/869598269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c0068d70 con 0x7f84c0102980 2024-04-21T10:29:07.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.441+0000 7f84b2ffd640 1 -- 172.21.15.26:0/869598269 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f84b80051d0 con 0x7f84c0102980 2024-04-21T10:29:07.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.442+0000 7f84b2ffd640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f84ac0a3c40 0x7f84ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:07.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.442+0000 7f84c5fce640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f84ac0a3c40 0x7f84ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:07.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.442+0000 7f84b2ffd640 1 -- 172.21.15.26:0/869598269 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f84b80d0670 con 0x7f84c0102980 2024-04-21T10:29:07.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.442+0000 7f84c5fce640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f84ac0a3c40 0x7f84ac0a6100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f84bc005fd0 tx=0x7f84bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:07.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.446+0000 7f84b2ffd640 1 -- 172.21.15.26:0/869598269 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84b8016030 con 0x7f84c0102980 2024-04-21T10:29:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.665+0000 7f84c8232640 1 -- 172.21.15.26:0/869598269 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f84c00697f0 con 0x7f84ac0a3c40 2024-04-21T10:29:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.670+0000 7f84b2ffd640 1 -- 172.21.15.26:0/869598269 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f84c00697f0 con 0x7f84ac0a3c40 2024-04-21T10:29:07.670 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/qa/sv_1/731d5979-300b-4e3c-938a-e127814d4722 2024-04-21T10:29:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.672+0000 7f84c8232640 1 -- 172.21.15.26:0/869598269 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f84ac0a3c40 msgr2=0x7f84ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.672+0000 7f84c8232640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f84ac0a3c40 0x7f84ac0a6100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f84bc005fd0 tx=0x7f84bc00b390 comp rx=0 tx=0).stop 2024-04-21T10:29:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.672+0000 7f84c8232640 1 -- 172.21.15.26:0/869598269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84c0102980 msgr2=0x7f84c0103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.672+0000 7f84c8232640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84c0102980 0x7f84c0103970 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f84b800eaa0 tx=0x7f84b800ef60 comp rx=0 tx=0).stop 2024-04-21T10:29:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.672+0000 7f84c8232640 1 -- 172.21.15.26:0/869598269 shutdown_connections 2024-04-21T10:29:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.673+0000 7f84c8232640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f84ac0a3c40 0x7f84ac0a6100 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.673+0000 7f84c8232640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84c0102980 0x7f84c0103970 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.673+0000 7f84c8232640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c0101fc0 0x7f84c0103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.673+0000 7f84c8232640 1 --2- 172.21.15.26:0/869598269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c0100e70 0x7f84c0104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.673+0000 7f84c8232640 1 -- 172.21.15.26:0/869598269 >> 172.21.15.26:0/869598269 conn(0x7f84c00fc3e0 msgr2=0x7f84c00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.673+0000 7f84c8232640 1 -- 172.21.15.26:0/869598269 shutdown_connections 2024-04-21T10:29:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.673+0000 7f84c8232640 1 -- 172.21.15.26:0/869598269 wait complete. 2024-04-21T10:29:07.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:29:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.784+0000 7fee28ab3640 1 -- 172.21.15.26:0/469257600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24105350 msgr2=0x7fee24107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.784+0000 7fee28ab3640 1 --2- 172.21.15.26:0/469257600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24105350 0x7fee24107740 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009900 tx=0x7fee0c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:07.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.784+0000 7fee28ab3640 1 -- 172.21.15.26:0/469257600 shutdown_connections 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.784+0000 7fee28ab3640 1 --2- 172.21.15.26:0/469257600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee24107c80 0x7fee2410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.784+0000 7fee28ab3640 1 --2- 172.21.15.26:0/469257600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24105350 0x7fee24107740 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.784+0000 7fee28ab3640 1 --2- 172.21.15.26:0/469257600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee24102a20 0x7fee24104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.784+0000 7fee28ab3640 1 -- 172.21.15.26:0/469257600 >> 172.21.15.26:0/469257600 conn(0x7fee240fc410 msgr2=0x7fee240fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.784+0000 7fee28ab3640 1 -- 172.21.15.26:0/469257600 shutdown_connections 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.784+0000 7fee28ab3640 1 -- 172.21.15.26:0/469257600 wait complete. 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.785+0000 7fee28ab3640 1 Processor -- start 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.785+0000 7fee28ab3640 1 -- start start 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.785+0000 7fee28ab3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee24102a20 0x7fee241981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.785+0000 7fee28ab3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee24105350 0x7fee24198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.785+0000 7fee28ab3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24107c80 0x7fee2419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.785+0000 7fee28ab3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee2410db30 con 0x7fee24107c80 2024-04-21T10:29:07.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.785+0000 7fee28ab3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee2410d9b0 con 0x7fee24105350 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.785+0000 7fee28ab3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fee2410dcb0 con 0x7fee24102a20 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.785+0000 7fee22d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24107c80 0x7fee2419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.785+0000 7fee22d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24107c80 0x7fee2419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58904/0 (socket says 172.21.15.26:58904) 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee22d9d640 1 -- 172.21.15.26:0/3284055230 learned_addr learned my addr 172.21.15.26:0/3284055230 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee2259c640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee24102a20 0x7fee241981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee21d9b640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee24105350 0x7fee24198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee22d9d640 1 -- 172.21.15.26:0/3284055230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee24102a20 msgr2=0x7fee241981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee22d9d640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee24102a20 0x7fee241981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee22d9d640 1 -- 172.21.15.26:0/3284055230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee24105350 msgr2=0x7fee24198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee22d9d640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee24105350 0x7fee24198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee22d9d640 1 -- 172.21.15.26:0/3284055230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2419fdf0 con 0x7fee24107c80 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee22d9d640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24107c80 0x7fee2419f780 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fee1400cad0 tx=0x7fee1400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:07.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee1b7fe640 1 -- 172.21.15.26:0/3284055230 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1400ae20 con 0x7fee24107c80 2024-04-21T10:29:07.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee1b7fe640 1 -- 172.21.15.26:0/3284055230 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee14005020 con 0x7fee24107c80 2024-04-21T10:29:07.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee28ab3640 1 -- 172.21.15.26:0/3284055230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee241a00e0 con 0x7fee24107c80 2024-04-21T10:29:07.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee1b7fe640 1 -- 172.21.15.26:0/3284055230 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1400f6b0 con 0x7fee24107c80 2024-04-21T10:29:07.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.786+0000 7fee28ab3640 1 -- 172.21.15.26:0/3284055230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee241a0600 con 0x7fee24107c80 2024-04-21T10:29:07.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.787+0000 7fee28ab3640 1 -- 172.21.15.26:0/3284055230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee240fdf00 con 0x7fee24107c80 2024-04-21T10:29:07.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.788+0000 7fee1b7fe640 1 -- 172.21.15.26:0/3284055230 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fee140051d0 con 0x7fee24107c80 2024-04-21T10:29:07.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.789+0000 7fee1b7fe640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fee100a3c40 0x7fee100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:07.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.789+0000 7fee2259c640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fee100a3c40 0x7fee100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:07.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.789+0000 7fee1b7fe640 1 -- 172.21.15.26:0/3284055230 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fee140d0510 con 0x7fee24107c80 2024-04-21T10:29:07.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.789+0000 7fee2259c640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fee100a3c40 0x7fee100a6100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fee1c005fd0 tx=0x7fee1c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:07.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:07.793+0000 7fee1b7fe640 1 -- 172.21.15.26:0/3284055230 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee14014030 con 0x7fee24107c80 2024-04-21T10:29:08.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.021+0000 7fee28ab3640 1 -- 172.21.15.26:0/3284055230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee24062b50 con 0x7fee24107c80 2024-04-21T10:29:08.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.022+0000 7fee1b7fe640 1 -- 172.21.15.26:0/3284055230 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11884 (secure 0 0 0) 0x7fee1409cf40 con 0x7fee24107c80 2024-04-21T10:29:08.022 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:29:08.022 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:29:05.806617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24478,24488,24496]},"id":1}]} 2024-04-21T10:29:08.022 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 20 2024-04-21T10:29:08.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.024+0000 7fee28ab3640 1 -- 172.21.15.26:0/3284055230 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fee100a3c40 msgr2=0x7fee100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.024+0000 7fee28ab3640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fee100a3c40 0x7fee100a6100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fee1c005fd0 tx=0x7fee1c005850 comp rx=0 tx=0).stop 2024-04-21T10:29:08.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.024+0000 7fee28ab3640 1 -- 172.21.15.26:0/3284055230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24107c80 msgr2=0x7fee2419f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.024+0000 7fee28ab3640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24107c80 0x7fee2419f780 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fee1400cad0 tx=0x7fee1400cf90 comp rx=0 tx=0).stop 2024-04-21T10:29:08.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.024+0000 7fee28ab3640 1 -- 172.21.15.26:0/3284055230 shutdown_connections 2024-04-21T10:29:08.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.024+0000 7fee28ab3640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fee100a3c40 0x7fee100a6100 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.024+0000 7fee28ab3640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24107c80 0x7fee2419f780 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.024+0000 7fee28ab3640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee24105350 0x7fee24198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.024+0000 7fee28ab3640 1 --2- 172.21.15.26:0/3284055230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee24102a20 0x7fee241981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.024+0000 7fee28ab3640 1 -- 172.21.15.26:0/3284055230 >> 172.21.15.26:0/3284055230 conn(0x7fee240fc410 msgr2=0x7fee24103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:08.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.024+0000 7fee28ab3640 1 -- 172.21.15.26:0/3284055230 shutdown_connections 2024-04-21T10:29:08.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.024+0000 7fee28ab3640 1 -- 172.21.15.26:0/3284055230 wait complete. 2024-04-21T10:29:08.035 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:29:08.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 -- 172.21.15.26:0/2956752510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39c102990 msgr2=0x7fa39c1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 --2- 172.21.15.26:0/2956752510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39c102990 0x7fa39c1092b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa394009900 tx=0x7fa39402bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 -- 172.21.15.26:0/2956752510 shutdown_connections 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 --2- 172.21.15.26:0/2956752510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39c102990 0x7fa39c1092b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 --2- 172.21.15.26:0/2956752510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa39c101fd0 0x7fa39c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 --2- 172.21.15.26:0/2956752510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa39c100e80 0x7fa39c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 -- 172.21.15.26:0/2956752510 >> 172.21.15.26:0/2956752510 conn(0x7fa39c0fc3f0 msgr2=0x7fa39c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 -- 172.21.15.26:0/2956752510 shutdown_connections 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 -- 172.21.15.26:0/2956752510 wait complete. 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 Processor -- start 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 -- start start 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa39c100e80 0x7fa39c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39c101fd0 0x7fa39c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa39c102990 0x7fa39c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.138+0000 7fa3a3c43640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c10db50 con 0x7fa39c101fd0 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a3c43640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c10d9d0 con 0x7fa39c102990 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a3c43640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c10dcd0 con 0x7fa39c100e80 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a11de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39c101fd0 0x7fa39c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a11de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39c101fd0 0x7fa39c1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58924/0 (socket says 172.21.15.26:58924) 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a11de640 1 -- 172.21.15.26:0/3608386613 learned_addr learned my addr 172.21.15.26:0/3608386613 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a19df640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa39c100e80 0x7fa39c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a11de640 1 -- 172.21.15.26:0/3608386613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa39c100e80 msgr2=0x7fa39c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a11de640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa39c100e80 0x7fa39c198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a11de640 1 -- 172.21.15.26:0/3608386613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa39c102990 msgr2=0x7fa39c19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a11de640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa39c102990 0x7fa39c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a11de640 1 -- 172.21.15.26:0/3608386613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c19fca0 con 0x7fa39c101fd0 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a11de640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39c101fd0 0x7fa39c1985c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00d770 tx=0x7fa38c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa392ffd640 1 -- 172.21.15.26:0/3608386613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c017070 con 0x7fa39c101fd0 2024-04-21T10:29:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa392ffd640 1 -- 172.21.15.26:0/3608386613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38c009c30 con 0x7fa39c101fd0 2024-04-21T10:29:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a3c43640 1 -- 172.21.15.26:0/3608386613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39c19ff90 con 0x7fa39c101fd0 2024-04-21T10:29:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa392ffd640 1 -- 172.21.15.26:0/3608386613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c003cc0 con 0x7fa39c101fd0 2024-04-21T10:29:08.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.139+0000 7fa3a3c43640 1 -- 172.21.15.26:0/3608386613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39c1a0450 con 0x7fa39c101fd0 2024-04-21T10:29:08.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.140+0000 7fa3a3c43640 1 -- 172.21.15.26:0/3608386613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa39c108130 con 0x7fa39c101fd0 2024-04-21T10:29:08.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.141+0000 7fa392ffd640 1 -- 172.21.15.26:0/3608386613 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa38c003e60 con 0x7fa39c101fd0 2024-04-21T10:29:08.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.142+0000 7fa392ffd640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa3880a3c40 0x7fa3880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:08.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.142+0000 7fa3a19df640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa3880a3c40 0x7fa3880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:08.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.142+0000 7fa392ffd640 1 -- 172.21.15.26:0/3608386613 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fa38c0cff40 con 0x7fa39c101fd0 2024-04-21T10:29:08.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.142+0000 7fa3a19df640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa3880a3c40 0x7fa3880a6100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa398008d20 tx=0x7fa3980082b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:08.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.146+0000 7fa392ffd640 1 -- 172.21.15.26:0/3608386613 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa38c09c870 con 0x7fa39c101fd0 2024-04-21T10:29:08.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:07 smithi190 ceph-mon[31423]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:08.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:07 smithi190 ceph-mon[31423]: pgmap v289: 97 pgs: 16 creating+peering, 81 active+clean; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 3.7 KiB/s wr, 10 op/s 2024-04-21T10:29:08.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:07 smithi190 ceph-mon[31423]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:08.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:07 smithi190 ceph-mon[31423]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:08.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:07 smithi190 ceph-mon[31423]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T10:29:08.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.334+0000 7fa3a3c43640 1 -- 172.21.15.26:0/3608386613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa39c02d7b0 con 0x7fa39c101fd0 2024-04-21T10:29:08.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.335+0000 7fa392ffd640 1 -- 172.21.15.26:0/3608386613 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22207 (secure 0 0 0) 0x7fa38c009de0 con 0x7fa39c101fd0 2024-04-21T10:29:08.335 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:29:08.335 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:29:02.925179+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:29:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.338+0000 7fa3a3c43640 1 -- 172.21.15.26:0/3608386613 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa3880a3c40 msgr2=0x7fa3880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.338+0000 7fa3a3c43640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa3880a3c40 0x7fa3880a6100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa398008d20 tx=0x7fa3980082b0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.338+0000 7fa3a3c43640 1 -- 172.21.15.26:0/3608386613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39c101fd0 msgr2=0x7fa39c1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.338+0000 7fa3a3c43640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39c101fd0 0x7fa39c1985c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00d770 tx=0x7fa38c00dc30 comp rx=0 tx=0).stop 2024-04-21T10:29:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.338+0000 7fa3a3c43640 1 -- 172.21.15.26:0/3608386613 shutdown_connections 2024-04-21T10:29:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.338+0000 7fa3a3c43640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa3880a3c40 0x7fa3880a6100 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.338+0000 7fa3a3c43640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa39c102990 0x7fa39c19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.338+0000 7fa3a3c43640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39c101fd0 0x7fa39c1985c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.338+0000 7fa3a3c43640 1 --2- 172.21.15.26:0/3608386613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa39c100e80 0x7fa39c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.338+0000 7fa3a3c43640 1 -- 172.21.15.26:0/3608386613 >> 172.21.15.26:0/3608386613 conn(0x7fa39c0fc3f0 msgr2=0x7fa39c105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.338+0000 7fa3a3c43640 1 -- 172.21.15.26:0/3608386613 shutdown_connections 2024-04-21T10:29:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.338+0000 7fa3a3c43640 1 -- 172.21.15.26:0/3608386613 wait complete. 2024-04-21T10:29:08.347 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:07 smithi026 ceph-mon[27965]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:08.347 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:07 smithi026 ceph-mon[27965]: pgmap v289: 97 pgs: 16 creating+peering, 81 active+clean; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 3.7 KiB/s wr, 10 op/s 2024-04-21T10:29:08.347 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:07 smithi026 ceph-mon[27965]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:08.347 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:07 smithi026 ceph-mon[27965]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:08.347 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:07 smithi026 ceph-mon[27965]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T10:29:08.349 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-21T10:29:08.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T10:29:08.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:07 smithi169 ceph-mon[31469]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:08.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:07 smithi169 ceph-mon[31469]: pgmap v289: 97 pgs: 16 creating+peering, 81 active+clean; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 3.7 KiB/s wr, 10 op/s 2024-04-21T10:29:08.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:07 smithi169 ceph-mon[31469]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:08.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:07 smithi169 ceph-mon[31469]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:08.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:07 smithi169 ceph-mon[31469]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T10:29:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.450+0000 7f9e68787640 1 -- 172.21.15.26:0/1630727705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60101f80 msgr2=0x7f9e601023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.450+0000 7f9e68787640 1 --2- 172.21.15.26:0/1630727705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60101f80 0x7f9e601023a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009ca0 tx=0x7f9e5802c0f0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 -- 172.21.15.26:0/1630727705 shutdown_connections 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 --2- 172.21.15.26:0/1630727705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e60103b30 0x7f9e6010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 --2- 172.21.15.26:0/1630727705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e60103170 0x7f9e601035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 --2- 172.21.15.26:0/1630727705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60101f80 0x7f9e601023a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 -- 172.21.15.26:0/1630727705 >> 172.21.15.26:0/1630727705 conn(0x7f9e600fd530 msgr2=0x7f9e600ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 -- 172.21.15.26:0/1630727705 shutdown_connections 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 -- 172.21.15.26:0/1630727705 wait complete. 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 Processor -- start 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 -- start start 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e60101f80 0x7f9e60198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60103170 0x7f9e60198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e60103b30 0x7f9e6019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6010dac0 con 0x7f9e60103170 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6010d940 con 0x7f9e60101f80 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.451+0000 7f9e68787640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6010dc40 con 0x7f9e60103b30 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e65d22640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60103170 0x7f9e60198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e65d22640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60103170 0x7f9e60198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58936/0 (socket says 172.21.15.26:58936) 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e65d22640 1 -- 172.21.15.26:0/795465714 learned_addr learned my addr 172.21.15.26:0/795465714 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e66523640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e60101f80 0x7f9e60198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e66d24640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e60103b30 0x7f9e6019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e65d22640 1 -- 172.21.15.26:0/795465714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e60103b30 msgr2=0x7f9e6019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e65d22640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e60103b30 0x7f9e6019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e65d22640 1 -- 172.21.15.26:0/795465714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e60101f80 msgr2=0x7f9e60198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e65d22640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e60101f80 0x7f9e60198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e65d22640 1 -- 172.21.15.26:0/795465714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6019fe20 con 0x7f9e60103170 2024-04-21T10:29:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e66d24640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e60103b30 0x7f9e6019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:29:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e65d22640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60103170 0x7f9e60198740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400e9f0 tx=0x7f9e5400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e5f7fe640 1 -- 172.21.15.26:0/795465714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5400cd80 con 0x7f9e60103170 2024-04-21T10:29:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e5f7fe640 1 -- 172.21.15.26:0/795465714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e540054d0 con 0x7f9e60103170 2024-04-21T10:29:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e68787640 1 -- 172.21.15.26:0/795465714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e601a0110 con 0x7f9e60103170 2024-04-21T10:29:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e5f7fe640 1 -- 172.21.15.26:0/795465714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54010690 con 0x7f9e60103170 2024-04-21T10:29:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.452+0000 7f9e68787640 1 -- 172.21.15.26:0/795465714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e601a0660 con 0x7f9e60103170 2024-04-21T10:29:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.453+0000 7f9e68787640 1 -- 172.21.15.26:0/795465714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e601093b0 con 0x7f9e60103170 2024-04-21T10:29:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.454+0000 7f9e5f7fe640 1 -- 172.21.15.26:0/795465714 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9e54004e90 con 0x7f9e60103170 2024-04-21T10:29:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.455+0000 7f9e5f7fe640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f9e480a3cc0 0x7f9e480a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.455+0000 7f9e66523640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f9e480a3cc0 0x7f9e480a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.455+0000 7f9e5f7fe640 1 -- 172.21.15.26:0/795465714 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f9e540d0820 con 0x7f9e60103170 2024-04-21T10:29:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.456+0000 7f9e66523640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f9e480a3cc0 0x7f9e480a6180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9e58032380 tx=0x7f9e58000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.460+0000 7f9e5f7fe640 1 -- 172.21.15.26:0/795465714 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e5409d250 con 0x7f9e60103170 2024-04-21T10:29:08.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.644+0000 7f9e68787640 1 -- 172.21.15.26:0/795465714 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e60062390 con 0x7f9e480a3cc0 2024-04-21T10:29:08.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.651+0000 7f9e5f7fe640 1 -- 172.21.15.26:0/795465714 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258757 (secure 0 0 0) 0x7f9e60062390 con 0x7f9e480a3cc0 2024-04-21T10:29:08.655 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:29:08.657 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-21T10:29:08.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.655+0000 7f9e68787640 1 -- 172.21.15.26:0/795465714 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f9e480a3cc0 msgr2=0x7f9e480a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.655+0000 7f9e68787640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f9e480a3cc0 0x7f9e480a6180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9e58032380 tx=0x7f9e58000f40 comp rx=0 tx=0).stop 2024-04-21T10:29:08.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.655+0000 7f9e68787640 1 -- 172.21.15.26:0/795465714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60103170 msgr2=0x7f9e60198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.655+0000 7f9e68787640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60103170 0x7f9e60198740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400e9f0 tx=0x7f9e5400edf0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.655+0000 7f9e68787640 1 -- 172.21.15.26:0/795465714 shutdown_connections 2024-04-21T10:29:08.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.655+0000 7f9e68787640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f9e480a3cc0 0x7f9e480a6180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.655+0000 7f9e68787640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e60103b30 0x7f9e6019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.655+0000 7f9e68787640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e60103170 0x7f9e60198740 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.655+0000 7f9e68787640 1 --2- 172.21.15.26:0/795465714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e60101f80 0x7f9e60198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.655+0000 7f9e68787640 1 -- 172.21.15.26:0/795465714 >> 172.21.15.26:0/795465714 conn(0x7f9e600fd530 msgr2=0x7f9e60106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.655+0000 7f9e68787640 1 -- 172.21.15.26:0/795465714 shutdown_connections 2024-04-21T10:29:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.655+0000 7f9e68787640 1 -- 172.21.15.26:0/795465714 wait complete. 2024-04-21T10:29:08.665 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-04-21T10:29:08.179286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":615381,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":169,"num_read_kb":164,"num_write":312,"num_write_kb":2999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":253,"ondisk_log_size":253,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362092,"kb_used_data":12928,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579732,"statfs":{"total":1151940427776,"available":1151569645568,"internally_reserved":0,"allocated":13238272,"data_stored":7157069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25013,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":33,"num_read_kb":45,"num_write":131,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.269046"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098382+0000","last_change":"2024-04-21T10:29:01.940996+0000","last_active":"2024-04-21T10:29:03.098382+0000","last_peered":"2024-04-21T10:29:03.098382+0000","last_clean":"2024-04-21T10:29:03.098382+0000","last_became_active":"2024-04-21T10:29:01.940662+0000","last_became_peered":"2024-04-21T10:29:01.940662+0000","last_unstale":"2024-04-21T10:29:03.098382+0000","last_undegraded":"2024-04-21T10:29:03.098382+0000","last_fullsized":"2024-04-21T10:29:03.098382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:16:01.434895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932733+0000","last_change":"2024-04-21T10:29:01.939238+0000","last_active":"2024-04-21T10:29:02.932733+0000","last_peered":"2024-04-21T10:29:02.932733+0000","last_clean":"2024-04-21T10:29:02.932733+0000","last_became_active":"2024-04-21T10:29:01.938907+0000","last_became_peered":"2024-04-21T10:29:01.938907+0000","last_unstale":"2024-04-21T10:29:02.932733+0000","last_undegraded":"2024-04-21T10:29:02.932733+0000","last_fullsized":"2024-04-21T10:29:02.932733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:09.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932734+0000","last_change":"2024-04-21T10:29:01.932338+0000","last_active":"2024-04-21T10:29:02.932734+0000","last_peered":"2024-04-21T10:29:02.932734+0000","last_clean":"2024-04-21T10:29:02.932734+0000","last_became_active":"2024-04-21T10:29:01.931916+0000","last_became_peered":"2024-04-21T10:29:01.931916+0000","last_unstale":"2024-04-21T10:29:02.932734+0000","last_undegraded":"2024-04-21T10:29:02.932734+0000","last_fullsized":"2024-04-21T10:29:02.932734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:52.859695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097400+0000","last_change":"2024-04-21T10:29:01.945572+0000","last_active":"2024-04-21T10:29:03.097400+0000","last_peered":"2024-04-21T10:29:03.097400+0000","last_clean":"2024-04-21T10:29:03.097400+0000","last_became_active":"2024-04-21T10:29:01.940114+0000","last_became_peered":"2024-04-21T10:29:01.940114+0000","last_unstale":"2024-04-21T10:29:03.097400+0000","last_undegraded":"2024-04-21T10:29:03.097400+0000","last_fullsized":"2024-04-21T10:29:03.097400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:21.935573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.934195+0000","last_change":"2024-04-21T10:29:01.946021+0000","last_active":"2024-04-21T10:29:02.934195+0000","last_peered":"2024-04-21T10:29:02.934195+0000","last_clean":"2024-04-21T10:29:02.934195+0000","last_became_active":"2024-04-21T10:29:01.945637+0000","last_became_peered":"2024-04-21T10:29:01.945637+0000","last_unstale":"2024-04-21T10:29:02.934195+0000","last_undegraded":"2024-04-21T10:29:02.934195+0000","last_fullsized":"2024-04-21T10:29:02.934195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:20.840494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097299+0000","last_change":"2024-04-21T10:29:01.935402+0000","last_active":"2024-04-21T10:29:03.097299+0000","last_peered":"2024-04-21T10:29:03.097299+0000","last_clean":"2024-04-21T10:29:03.097299+0000","last_became_active":"2024-04-21T10:29:01.934994+0000","last_became_peered":"2024-04-21T10:29:01.934994+0000","last_unstale":"2024-04-21T10:29:03.097299+0000","last_undegraded":"2024-04-21T10:29:03.097299+0000","last_fullsized":"2024-04-21T10:29:03.097299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:18.320530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.937751+0000","last_change":"2024-04-21T10:29:01.943185+0000","last_active":"2024-04-21T10:29:02.937751+0000","last_peered":"2024-04-21T10:29:02.937751+0000","last_clean":"2024-04-21T10:29:02.937751+0000","last_became_active":"2024-04-21T10:29:01.942805+0000","last_became_peered":"2024-04-21T10:29:01.942805+0000","last_unstale":"2024-04-21T10:29:02.937751+0000","last_undegraded":"2024-04-21T10:29:02.937751+0000","last_fullsized":"2024-04-21T10:29:02.937751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:06.776472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096682+0000","last_change":"2024-04-21T10:29:01.940995+0000","last_active":"2024-04-21T10:29:03.096682+0000","last_peered":"2024-04-21T10:29:03.096682+0000","last_clean":"2024-04-21T10:29:03.096682+0000","last_became_active":"2024-04-21T10:29:01.939173+0000","last_became_peered":"2024-04-21T10:29:01.939173+0000","last_unstale":"2024-04-21T10:29:03.096682+0000","last_undegraded":"2024-04-21T10:29:03.096682+0000","last_fullsized":"2024-04-21T10:29:03.096682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.740147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098178+0000","last_change":"2024-04-21T10:29:01.944284+0000","last_active":"2024-04-21T10:29:03.098178+0000","last_peered":"2024-04-21T10:29:03.098178+0000","last_clean":"2024-04-21T10:29:03.098178+0000","last_became_active":"2024-04-21T10:29:01.943965+0000","last_became_peered":"2024-04-21T10:29:01.943965+0000","last_unstale":"2024-04-21T10:29:03.098178+0000","last_undegraded":"2024-04-21T10:29:03.098178+0000","last_fullsized":"2024-04-21T10:29:03.098178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:08:54.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098288+0000","last_change":"2024-04-21T10:29:01.936250+0000","last_active":"2024-04-21T10:29:03.098288+0000","last_peered":"2024-04-21T10:29:03.098288+0000","last_clean":"2024-04-21T10:29:03.098288+0000","last_became_active":"2024-04-21T10:29:01.935904+0000","last_became_peered":"2024-04-21T10:29:01.935904+0000","last_unstale":"2024-04-21T10:29:03.098288+0000","last_undegraded":"2024-04-21T10:29:03.098288+0000","last_fullsized":"2024-04-21T10:29:03.098288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:35.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.938138+0000","last_change":"2024-04-21T10:29:01.946550+0000","last_active":"2024-04-21T10:29:02.938138+0000","last_peered":"2024-04-21T10:29:02.938138+0000","last_clean":"2024-04-21T10:29:02.938138+0000","last_became_active":"2024-04-21T10:29:01.946170+0000","last_became_peered":"2024-04-21T10:29:01.946170+0000","last_unstale":"2024-04-21T10:29:02.938138+0000","last_undegraded":"2024-04-21T10:29:02.938138+0000","last_fullsized":"2024-04-21T10:29:02.938138+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:25:47.671821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097217+0000","last_change":"2024-04-21T10:29:01.946036+0000","last_active":"2024-04-21T10:29:03.097217+0000","last_peered":"2024-04-21T10:29:03.097217+0000","last_clean":"2024-04-21T10:29:03.097217+0000","last_became_active":"2024-04-21T10:29:01.940705+0000","last_became_peered":"2024-04-21T10:29:01.940705+0000","last_unstale":"2024-04-21T10:29:03.097217+0000","last_undegraded":"2024-04-21T10:29:03.097217+0000","last_fullsized":"2024-04-21T10:29:03.097217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:11:33.196059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097218+0000","last_change":"2024-04-21T10:29:01.940438+0000","last_active":"2024-04-21T10:29:03.097218+0000","last_peered":"2024-04-21T10:29:03.097218+0000","last_clean":"2024-04-21T10:29:03.097218+0000","last_became_active":"2024-04-21T10:29:01.940085+0000","last_became_peered":"2024-04-21T10:29:01.940085+0000","last_unstale":"2024-04-21T10:29:03.097218+0000","last_undegraded":"2024-04-21T10:29:03.097218+0000","last_fullsized":"2024-04-21T10:29:03.097218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:22:47.828858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932399+0000","last_change":"2024-04-21T10:29:01.941667+0000","last_active":"2024-04-21T10:29:02.932399+0000","last_peered":"2024-04-21T10:29:02.932399+0000","last_clean":"2024-04-21T10:29:02.932399+0000","last_became_active":"2024-04-21T10:29:01.940745+0000","last_became_peered":"2024-04-21T10:29:01.940745+0000","last_unstale":"2024-04-21T10:29:02.932399+0000","last_undegraded":"2024-04-21T10:29:02.932399+0000","last_fullsized":"2024-04-21T10:29:02.932399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:45:51.825780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932401+0000","last_change":"2024-04-21T10:29:01.947771+0000","last_active":"2024-04-21T10:29:02.932401+0000","last_peered":"2024-04-21T10:29:02.932401+0000","last_clean":"2024-04-21T10:29:02.932401+0000","last_became_active":"2024-04-21T10:29:01.946818+0000","last_became_peered":"2024-04-21T10:29:01.946818+0000","last_unstale":"2024-04-21T10:29:02.932401+0000","last_undegraded":"2024-04-21T10:29:02.932401+0000","last_fullsized":"2024-04-21T10:29:02.932401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:18:42.159152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096886+0000","last_change":"2024-04-21T10:29:01.942499+0000","last_active":"2024-04-21T10:29:03.096886+0000","last_peered":"2024-04-21T10:29:03.096886+0000","last_clean":"2024-04-21T10:29:03.096886+0000","last_became_active":"2024-04-21T10:29:01.942136+0000","last_became_peered":"2024-04-21T10:29:01.942136+0000","last_unstale":"2024-04-21T10:29:03.096886+0000","last_undegraded":"2024-04-21T10:29:03.096886+0000","last_fullsized":"2024-04-21T10:29:03.096886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:34:11.822230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098546+0000","last_change":"2024-04-21T10:29:01.948822+0000","last_active":"2024-04-21T10:29:03.098546+0000","last_peered":"2024-04-21T10:29:03.098546+0000","last_clean":"2024-04-21T10:29:03.098546+0000","last_became_active":"2024-04-21T10:29:01.948492+0000","last_became_peered":"2024-04-21T10:29:01.948492+0000","last_unstale":"2024-04-21T10:29:03.098546+0000","last_undegraded":"2024-04-21T10:29:03.098546+0000","last_fullsized":"2024-04-21T10:29:03.098546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:06.992585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096820+0000","last_change":"2024-04-21T10:29:01.931265+0000","last_active":"2024-04-21T10:29:03.096820+0000","last_peered":"2024-04-21T10:29:03.096820+0000","last_clean":"2024-04-21T10:29:03.096820+0000","last_became_active":"2024-04-21T10:29:01.930877+0000","last_became_peered":"2024-04-21T10:29:01.930877+0000","last_unstale":"2024-04-21T10:29:03.096820+0000","last_undegraded":"2024-04-21T10:29:03.096820+0000","last_fullsized":"2024-04-21T10:29:03.096820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:49:00.951226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098552+0000","last_change":"2024-04-21T10:29:01.941718+0000","last_active":"2024-04-21T10:29:03.098552+0000","last_peered":"2024-04-21T10:29:03.098552+0000","last_clean":"2024-04-21T10:29:03.098552+0000","last_became_active":"2024-04-21T10:29:01.941386+0000","last_became_peered":"2024-04-21T10:29:01.941386+0000","last_unstale":"2024-04-21T10:29:03.098552+0000","last_undegraded":"2024-04-21T10:29:03.098552+0000","last_fullsized":"2024-04-21T10:29:03.098552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:51:29.847961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.814006+0000","last_change":"2024-04-21T10:29:00.923670+0000","last_active":"2024-04-21T10:29:04.814006+0000","last_peered":"2024-04-21T10:29:04.814006+0000","last_clean":"2024-04-21T10:29:04.814006+0000","last_became_active":"2024-04-21T10:29:00.923100+0000","last_became_peered":"2024-04-21T10:29:00.923100+0000","last_unstale":"2024-04-21T10:29:04.814006+0000","last_undegraded":"2024-04-21T10:29:04.814006+0000","last_fullsized":"2024-04-21T10:29:04.814006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:45.692961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096884+0000","last_change":"2024-04-21T10:29:01.938131+0000","last_active":"2024-04-21T10:29:03.096884+0000","last_peered":"2024-04-21T10:29:03.096884+0000","last_clean":"2024-04-21T10:29:03.096884+0000","last_became_active":"2024-04-21T10:29:01.937752+0000","last_became_peered":"2024-04-21T10:29:01.937752+0000","last_unstale":"2024-04-21T10:29:03.096884+0000","last_undegraded":"2024-04-21T10:29:03.096884+0000","last_fullsized":"2024-04-21T10:29:03.096884+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:06:49.383554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096995+0000","last_change":"2024-04-21T10:29:00.929877+0000","last_active":"2024-04-21T10:29:03.096995+0000","last_peered":"2024-04-21T10:29:03.096995+0000","last_clean":"2024-04-21T10:29:03.096995+0000","last_became_active":"2024-04-21T10:29:00.929564+0000","last_became_peered":"2024-04-21T10:29:00.929564+0000","last_unstale":"2024-04-21T10:29:03.096995+0000","last_undegraded":"2024-04-21T10:29:03.096995+0000","last_fullsized":"2024-04-21T10:29:03.096995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:35.617533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932928+0000","last_change":"2024-04-21T10:29:01.942782+0000","last_active":"2024-04-21T10:29:02.932928+0000","last_peered":"2024-04-21T10:29:02.932928+0000","last_clean":"2024-04-21T10:29:02.932928+0000","last_became_active":"2024-04-21T10:29:01.940501+0000","last_became_peered":"2024-04-21T10:29:01.940501+0000","last_unstale":"2024-04-21T10:29:02.932928+0000","last_undegraded":"2024-04-21T10:29:02.932928+0000","last_fullsized":"2024-04-21T10:29:02.932928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:04:03.431016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.814189+0000","last_change":"2024-04-21T10:29:00.924395+0000","last_active":"2024-04-21T10:29:04.814189+0000","last_peered":"2024-04-21T10:29:04.814189+0000","last_clean":"2024-04-21T10:29:04.814189+0000","last_became_active":"2024-04-21T10:29:00.924045+0000","last_became_peered":"2024-04-21T10:29:00.924045+0000","last_unstale":"2024-04-21T10:29:04.814189+0000","last_undegraded":"2024-04-21T10:29:04.814189+0000","last_fullsized":"2024-04-21T10:29:04.814189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:08:53.922213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097980+0000","last_change":"2024-04-21T10:29:01.934508+0000","last_active":"2024-04-21T10:29:03.097980+0000","last_peered":"2024-04-21T10:29:03.097980+0000","last_clean":"2024-04-21T10:29:03.097980+0000","last_became_active":"2024-04-21T10:29:01.934051+0000","last_became_peered":"2024-04-21T10:29:01.934051+0000","last_unstale":"2024-04-21T10:29:03.097980+0000","last_undegraded":"2024-04-21T10:29:03.097980+0000","last_fullsized":"2024-04-21T10:29:03.097980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:49:27.686603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.934347+0000","last_change":"2024-04-21T10:29:00.925850+0000","last_active":"2024-04-21T10:29:02.934347+0000","last_peered":"2024-04-21T10:29:02.934347+0000","last_clean":"2024-04-21T10:29:02.934347+0000","last_became_active":"2024-04-21T10:29:00.925521+0000","last_became_peered":"2024-04-21T10:29:00.925521+0000","last_unstale":"2024-04-21T10:29:02.934347+0000","last_undegraded":"2024-04-21T10:29:02.934347+0000","last_fullsized":"2024-04-21T10:29:02.934347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:36:55.403413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098269+0000","last_change":"2024-04-21T10:29:01.930046+0000","last_active":"2024-04-21T10:29:03.098269+0000","last_peered":"2024-04-21T10:29:03.098269+0000","last_clean":"2024-04-21T10:29:03.098269+0000","last_became_active":"2024-04-21T10:29:01.929736+0000","last_became_peered":"2024-04-21T10:29:01.929736+0000","last_unstale":"2024-04-21T10:29:03.098269+0000","last_undegraded":"2024-04-21T10:29:03.098269+0000","last_fullsized":"2024-04-21T10:29:03.098269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:11:10.833267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:05.816158+0000","last_change":"2024-04-21T10:29:00.927936+0000","last_active":"2024-04-21T10:29:05.816158+0000","last_peered":"2024-04-21T10:29:05.816158+0000","last_clean":"2024-04-21T10:29:05.816158+0000","last_became_active":"2024-04-21T10:29:00.927352+0000","last_became_peered":"2024-04-21T10:29:00.927352+0000","last_unstale":"2024-04-21T10:29:05.816158+0000","last_undegraded":"2024-04-21T10:29:05.816158+0000","last_fullsized":"2024-04-21T10:29:05.816158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:14:10.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.934244+0000","last_change":"2024-04-21T10:29:01.943389+0000","last_active":"2024-04-21T10:29:02.934244+0000","last_peered":"2024-04-21T10:29:02.934244+0000","last_clean":"2024-04-21T10:29:02.934244+0000","last_became_active":"2024-04-21T10:29:01.940386+0000","last_became_peered":"2024-04-21T10:29:01.940386+0000","last_unstale":"2024-04-21T10:29:02.934244+0000","last_undegraded":"2024-04-21T10:29:02.934244+0000","last_fullsized":"2024-04-21T10:29:02.934244+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:11:54.894785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.813157+0000","last_change":"2024-04-21T10:29:00.925945+0000","last_active":"2024-04-21T10:29:04.813157+0000","last_peered":"2024-04-21T10:29:04.813157+0000","last_clean":"2024-04-21T10:29:04.813157+0000","last_became_active":"2024-04-21T10:29:00.925641+0000","last_became_peered":"2024-04-21T10:29:00.925641+0000","last_unstale":"2024-04-21T10:29:04.813157+0000","last_undegraded":"2024-04-21T10:29:04.813157+0000","last_fullsized":"2024-04-21T10:29:04.813157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:44:18.204033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097636+0000","last_change":"2024-04-21T10:29:01.918627+0000","last_active":"2024-04-21T10:29:03.097636+0000","last_peered":"2024-04-21T10:29:03.097636+0000","last_clean":"2024-04-21T10:29:03.097636+0000","last_became_active":"2024-04-21T10:29:01.917993+0000","last_became_peered":"2024-04-21T10:29:01.917993+0000","last_unstale":"2024-04-21T10:29:03.097636+0000","last_undegraded":"2024-04-21T10:29:03.097636+0000","last_fullsized":"2024-04-21T10:29:03.097636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:58.891918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.813855+0000","last_change":"2024-04-21T10:29:00.923613+0000","last_active":"2024-04-21T10:29:04.813855+0000","last_peered":"2024-04-21T10:29:04.813855+0000","last_clean":"2024-04-21T10:29:04.813855+0000","last_became_active":"2024-04-21T10:29:00.923001+0000","last_became_peered":"2024-04-21T10:29:00.923001+0000","last_unstale":"2024-04-21T10:29:04.813855+0000","last_undegraded":"2024-04-21T10:29:04.813855+0000","last_fullsized":"2024-04-21T10:29:04.813855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:44:53.593903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096485+0000","last_change":"2024-04-21T10:29:01.934974+0000","last_active":"2024-04-21T10:29:03.096485+0000","last_peered":"2024-04-21T10:29:03.096485+0000","last_clean":"2024-04-21T10:29:03.096485+0000","last_became_active":"2024-04-21T10:29:01.932443+0000","last_became_peered":"2024-04-21T10:29:01.932443+0000","last_unstale":"2024-04-21T10:29:03.096485+0000","last_undegraded":"2024-04-21T10:29:03.096485+0000","last_fullsized":"2024-04-21T10:29:03.096485+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:24:47.190142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097123+0000","last_change":"2024-04-21T10:29:00.928179+0000","last_active":"2024-04-21T10:29:03.097123+0000","last_peered":"2024-04-21T10:29:03.097123+0000","last_clean":"2024-04-21T10:29:03.097123+0000","last_became_active":"2024-04-21T10:29:00.927588+0000","last_became_peered":"2024-04-21T10:29:00.927588+0000","last_unstale":"2024-04-21T10:29:03.097123+0000","last_undegraded":"2024-04-21T10:29:03.097123+0000","last_fullsized":"2024-04-21T10:29:03.097123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:04:41.081424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097143+0000","last_change":"2024-04-21T10:29:01.944421+0000","last_active":"2024-04-21T10:29:03.097143+0000","last_peered":"2024-04-21T10:29:03.097143+0000","last_clean":"2024-04-21T10:29:03.097143+0000","last_became_active":"2024-04-21T10:29:01.943572+0000","last_became_peered":"2024-04-21T10:29:01.943572+0000","last_unstale":"2024-04-21T10:29:03.097143+0000","last_undegraded":"2024-04-21T10:29:03.097143+0000","last_fullsized":"2024-04-21T10:29:03.097143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:34:01.690600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:05.814317+0000","last_change":"2024-04-21T10:29:00.924406+0000","last_active":"2024-04-21T10:29:05.814317+0000","last_peered":"2024-04-21T10:29:05.814317+0000","last_clean":"2024-04-21T10:29:05.814317+0000","last_became_active":"2024-04-21T10:29:00.924094+0000","last_became_peered":"2024-04-21T10:29:00.924094+0000","last_unstale":"2024-04-21T10:29:05.814317+0000","last_undegraded":"2024-04-21T10:29:05.814317+0000","last_fullsized":"2024-04-21T10:29:05.814317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:44.673751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098275+0000","last_change":"2024-04-21T10:29:01.933048+0000","last_active":"2024-04-21T10:29:03.098275+0000","last_peered":"2024-04-21T10:29:03.098275+0000","last_clean":"2024-04-21T10:29:03.098275+0000","last_became_active":"2024-04-21T10:29:01.932659+0000","last_became_peered":"2024-04-21T10:29:01.932659+0000","last_unstale":"2024-04-21T10:29:03.098275+0000","last_undegraded":"2024-04-21T10:29:03.098275+0000","last_fullsized":"2024-04-21T10:29:03.098275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:22:13.712112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.806580+0000","last_change":"2024-04-21T10:29:00.923020+0000","last_active":"2024-04-21T10:29:03.806580+0000","last_peered":"2024-04-21T10:29:03.806580+0000","last_clean":"2024-04-21T10:29:03.806580+0000","last_became_active":"2024-04-21T10:29:00.922678+0000","last_became_peered":"2024-04-21T10:29:00.922678+0000","last_unstale":"2024-04-21T10:29:03.806580+0000","last_undegraded":"2024-04-21T10:29:03.806580+0000","last_fullsized":"2024-04-21T10:29:03.806580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:13:34.698082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932938+0000","last_change":"2024-04-21T10:29:01.947833+0000","last_active":"2024-04-21T10:29:02.932938+0000","last_peered":"2024-04-21T10:29:02.932938+0000","last_clean":"2024-04-21T10:29:02.932938+0000","last_became_active":"2024-04-21T10:29:01.947351+0000","last_became_peered":"2024-04-21T10:29:01.947351+0000","last_unstale":"2024-04-21T10:29:02.932938+0000","last_undegraded":"2024-04-21T10:29:02.932938+0000","last_fullsized":"2024-04-21T10:29:02.932938+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:54.538731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.805072+0000","last_change":"2024-04-21T10:29:00.932309+0000","last_active":"2024-04-21T10:29:03.805072+0000","last_peered":"2024-04-21T10:29:03.805072+0000","last_clean":"2024-04-21T10:29:03.805072+0000","last_became_active":"2024-04-21T10:29:00.931935+0000","last_became_peered":"2024-04-21T10:29:00.931935+0000","last_unstale":"2024-04-21T10:29:03.805072+0000","last_undegraded":"2024-04-21T10:29:03.805072+0000","last_fullsized":"2024-04-21T10:29:03.805072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:37:37.604381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097800+0000","last_change":"2024-04-21T10:29:01.946543+0000","last_active":"2024-04-21T10:29:03.097800+0000","last_peered":"2024-04-21T10:29:03.097800+0000","last_clean":"2024-04-21T10:29:03.097800+0000","last_became_active":"2024-04-21T10:29:01.946210+0000","last_became_peered":"2024-04-21T10:29:01.946210+0000","last_unstale":"2024-04-21T10:29:03.097800+0000","last_undegraded":"2024-04-21T10:29:03.097800+0000","last_fullsized":"2024-04-21T10:29:03.097800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:49.368851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:06.817758+0000","last_change":"2024-04-21T10:29:00.924109+0000","last_active":"2024-04-21T10:29:06.817758+0000","last_peered":"2024-04-21T10:29:06.817758+0000","last_clean":"2024-04-21T10:29:06.817758+0000","last_became_active":"2024-04-21T10:29:00.923735+0000","last_became_peered":"2024-04-21T10:29:00.923735+0000","last_unstale":"2024-04-21T10:29:06.817758+0000","last_undegraded":"2024-04-21T10:29:06.817758+0000","last_fullsized":"2024-04-21T10:29:06.817758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:13:54.015774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":23,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.934404+0000","last_change":"2024-04-21T10:29:01.937211+0000","last_active":"2024-04-21T10:29:02.934404+0000","last_peered":"2024-04-21T10:29:02.934404+0000","last_clean":"2024-04-21T10:29:02.934404+0000","last_became_active":"2024-04-21T10:29:01.936891+0000","last_became_peered":"2024-04-21T10:29:01.936891+0000","last_unstale":"2024-04-21T10:29:02.934404+0000","last_undegraded":"2024-04-21T10:29:02.934404+0000","last_fullsized":"2024-04-21T10:29:02.934404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:26.062236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.816416+0000","last_change":"2024-04-21T10:29:00.920626+0000","last_active":"2024-04-21T10:29:04.816416+0000","last_peered":"2024-04-21T10:29:04.816416+0000","last_clean":"2024-04-21T10:29:04.816416+0000","last_became_active":"2024-04-21T10:29:00.920317+0000","last_became_peered":"2024-04-21T10:29:00.920317+0000","last_unstale":"2024-04-21T10:29:04.816416+0000","last_undegraded":"2024-04-21T10:29:04.816416+0000","last_fullsized":"2024-04-21T10:29:04.816416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:41:18.373403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097461+0000","last_change":"2024-04-21T10:29:01.942494+0000","last_active":"2024-04-21T10:29:03.097461+0000","last_peered":"2024-04-21T10:29:03.097461+0000","last_clean":"2024-04-21T10:29:03.097461+0000","last_became_active":"2024-04-21T10:29:01.942151+0000","last_became_peered":"2024-04-21T10:29:01.942151+0000","last_unstale":"2024-04-21T10:29:03.097461+0000","last_undegraded":"2024-04-21T10:29:03.097461+0000","last_fullsized":"2024-04-21T10:29:03.097461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:39.071158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.807417+0000","last_change":"2024-04-21T10:29:00.928221+0000","last_active":"2024-04-21T10:29:03.807417+0000","last_peered":"2024-04-21T10:29:03.807417+0000","last_clean":"2024-04-21T10:29:03.807417+0000","last_became_active":"2024-04-21T10:29:00.927873+0000","last_became_peered":"2024-04-21T10:29:00.927873+0000","last_unstale":"2024-04-21T10:29:03.807417+0000","last_undegraded":"2024-04-21T10:29:03.807417+0000","last_fullsized":"2024-04-21T10:29:03.807417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:14.898456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.938125+0000","last_change":"2024-04-21T10:29:01.930375+0000","last_active":"2024-04-21T10:29:02.938125+0000","last_peered":"2024-04-21T10:29:02.938125+0000","last_clean":"2024-04-21T10:29:02.938125+0000","last_became_active":"2024-04-21T10:29:01.930009+0000","last_became_peered":"2024-04-21T10:29:01.930009+0000","last_unstale":"2024-04-21T10:29:02.938125+0000","last_undegraded":"2024-04-21T10:29:02.938125+0000","last_fullsized":"2024-04-21T10:29:02.938125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:34.111797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:06.818158+0000","last_change":"2024-04-21T10:29:00.922981+0000","last_active":"2024-04-21T10:29:06.818158+0000","last_peered":"2024-04-21T10:29:06.818158+0000","last_clean":"2024-04-21T10:29:06.818158+0000","last_became_active":"2024-04-21T10:29:00.922668+0000","last_became_peered":"2024-04-21T10:29:00.922668+0000","last_unstale":"2024-04-21T10:29:06.818158+0000","last_undegraded":"2024-04-21T10:29:06.818158+0000","last_fullsized":"2024-04-21T10:29:06.818158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:47.109375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.937635+0000","last_change":"2024-04-21T10:29:01.942530+0000","last_active":"2024-04-21T10:29:02.937635+0000","last_peered":"2024-04-21T10:29:02.937635+0000","last_clean":"2024-04-21T10:29:02.937635+0000","last_became_active":"2024-04-21T10:29:01.942162+0000","last_became_peered":"2024-04-21T10:29:01.942162+0000","last_unstale":"2024-04-21T10:29:02.937635+0000","last_undegraded":"2024-04-21T10:29:02.937635+0000","last_fullsized":"2024-04-21T10:29:02.937635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:46:19.138930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.815197+0000","last_change":"2024-04-21T10:29:00.921104+0000","last_active":"2024-04-21T10:29:04.815197+0000","last_peered":"2024-04-21T10:29:04.815197+0000","last_clean":"2024-04-21T10:29:04.815197+0000","last_became_active":"2024-04-21T10:29:00.920772+0000","last_became_peered":"2024-04-21T10:29:00.920772+0000","last_unstale":"2024-04-21T10:29:04.815197+0000","last_undegraded":"2024-04-21T10:29:04.815197+0000","last_fullsized":"2024-04-21T10:29:04.815197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:23:55.738067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097433+0000","last_change":"2024-04-21T10:29:01.929696+0000","last_active":"2024-04-21T10:29:03.097433+0000","last_peered":"2024-04-21T10:29:03.097433+0000","last_clean":"2024-04-21T10:29:03.097433+0000","last_became_active":"2024-04-21T10:29:01.929328+0000","last_became_peered":"2024-04-21T10:29:01.929328+0000","last_unstale":"2024-04-21T10:29:03.097433+0000","last_undegraded":"2024-04-21T10:29:03.097433+0000","last_fullsized":"2024-04-21T10:29:03.097433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:55:42.735595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:05.808476+0000","last_change":"2024-04-21T10:29:00.925981+0000","last_active":"2024-04-21T10:29:05.808476+0000","last_peered":"2024-04-21T10:29:05.808476+0000","last_clean":"2024-04-21T10:29:05.808476+0000","last_became_active":"2024-04-21T10:29:00.925560+0000","last_became_peered":"2024-04-21T10:29:00.925560+0000","last_unstale":"2024-04-21T10:29:05.808476+0000","last_undegraded":"2024-04-21T10:29:05.808476+0000","last_fullsized":"2024-04-21T10:29:05.808476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:37.467522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097893+0000","last_change":"2024-04-21T10:29:01.941961+0000","last_active":"2024-04-21T10:29:03.097893+0000","last_peered":"2024-04-21T10:29:03.097893+0000","last_clean":"2024-04-21T10:29:03.097893+0000","last_became_active":"2024-04-21T10:29:01.941649+0000","last_became_peered":"2024-04-21T10:29:01.941649+0000","last_unstale":"2024-04-21T10:29:03.097893+0000","last_undegraded":"2024-04-21T10:29:03.097893+0000","last_fullsized":"2024-04-21T10:29:03.097893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:00:56.803856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096656+0000","last_change":"2024-04-21T10:29:00.929504+0000","last_active":"2024-04-21T10:29:03.096656+0000","last_peered":"2024-04-21T10:29:03.096656+0000","last_clean":"2024-04-21T10:29:03.096656+0000","last_became_active":"2024-04-21T10:29:00.929172+0000","last_became_peered":"2024-04-21T10:29:00.929172+0000","last_unstale":"2024-04-21T10:29:03.096656+0000","last_undegraded":"2024-04-21T10:29:03.096656+0000","last_fullsized":"2024-04-21T10:29:03.096656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:40.512175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098134+0000","last_change":"2024-04-21T10:29:01.939333+0000","last_active":"2024-04-21T10:29:03.098134+0000","last_peered":"2024-04-21T10:29:03.098134+0000","last_clean":"2024-04-21T10:29:03.098134+0000","last_became_active":"2024-04-21T10:29:01.938924+0000","last_became_peered":"2024-04-21T10:29:01.938924+0000","last_unstale":"2024-04-21T10:29:03.098134+0000","last_undegraded":"2024-04-21T10:29:03.098134+0000","last_fullsized":"2024-04-21T10:29:03.098134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:20:01.287604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.937646+0000","last_change":"2024-04-21T10:29:00.927455+0000","last_active":"2024-04-21T10:29:02.937646+0000","last_peered":"2024-04-21T10:29:02.937646+0000","last_clean":"2024-04-21T10:29:02.937646+0000","last_became_active":"2024-04-21T10:29:00.927180+0000","last_became_peered":"2024-04-21T10:29:00.927180+0000","last_unstale":"2024-04-21T10:29:02.937646+0000","last_undegraded":"2024-04-21T10:29:02.937646+0000","last_fullsized":"2024-04-21T10:29:02.937646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:11:57.166607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097261+0000","last_change":"2024-04-21T10:29:01.937646+0000","last_active":"2024-04-21T10:29:03.097261+0000","last_peered":"2024-04-21T10:29:03.097261+0000","last_clean":"2024-04-21T10:29:03.097261+0000","last_became_active":"2024-04-21T10:29:01.937216+0000","last_became_peered":"2024-04-21T10:29:01.937216+0000","last_unstale":"2024-04-21T10:29:03.097261+0000","last_undegraded":"2024-04-21T10:29:03.097261+0000","last_fullsized":"2024-04-21T10:29:03.097261+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:38.265252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097258+0000","last_change":"2024-04-21T10:29:01.926860+0000","last_active":"2024-04-21T10:29:03.097258+0000","last_peered":"2024-04-21T10:29:03.097258+0000","last_clean":"2024-04-21T10:29:03.097258+0000","last_became_active":"2024-04-21T10:29:01.925951+0000","last_became_peered":"2024-04-21T10:29:01.925951+0000","last_unstale":"2024-04-21T10:29:03.097258+0000","last_undegraded":"2024-04-21T10:29:03.097258+0000","last_fullsized":"2024-04-21T10:29:03.097258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:01:55.127301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.814347+0000","last_change":"2024-04-21T10:29:00.926502+0000","last_active":"2024-04-21T10:29:04.814347+0000","last_peered":"2024-04-21T10:29:04.814347+0000","last_clean":"2024-04-21T10:29:04.814347+0000","last_became_active":"2024-04-21T10:29:00.926175+0000","last_became_peered":"2024-04-21T10:29:00.926175+0000","last_unstale":"2024-04-21T10:29:04.814347+0000","last_undegraded":"2024-04-21T10:29:04.814347+0000","last_fullsized":"2024-04-21T10:29:04.814347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:40:34.774085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097956+0000","last_change":"2024-04-21T10:29:01.926369+0000","last_active":"2024-04-21T10:29:03.097956+0000","last_peered":"2024-04-21T10:29:03.097956+0000","last_clean":"2024-04-21T10:29:03.097956+0000","last_became_active":"2024-04-21T10:29:01.926011+0000","last_became_peered":"2024-04-21T10:29:01.926011+0000","last_unstale":"2024-04-21T10:29:03.097956+0000","last_undegraded":"2024-04-21T10:29:03.097956+0000","last_fullsized":"2024-04-21T10:29:03.097956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:43:18.273057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:05.812932+0000","last_change":"2024-04-21T10:29:00.921748+0000","last_active":"2024-04-21T10:29:05.812932+0000","last_peered":"2024-04-21T10:29:05.812932+0000","last_clean":"2024-04-21T10:29:05.812932+0000","last_became_active":"2024-04-21T10:29:00.921408+0000","last_became_peered":"2024-04-21T10:29:00.921408+0000","last_unstale":"2024-04-21T10:29:05.812932+0000","last_undegraded":"2024-04-21T10:29:05.812932+0000","last_fullsized":"2024-04-21T10:29:05.812932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:52:27.606280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.937503+0000","last_change":"2024-04-21T10:27:49.787009+0000","last_active":"2024-04-21T10:29:02.937503+0000","last_peered":"2024-04-21T10:29:02.937503+0000","last_clean":"2024-04-21T10:29:02.937503+0000","last_became_active":"2024-04-21T10:27:49.786530+0000","last_became_peered":"2024-04-21T10:27:49.786530+0000","last_unstale":"2024-04-21T10:29:02.937503+0000","last_undegraded":"2024-04-21T10:29:02.937503+0000","last_fullsized":"2024-04-21T10:29:02.937503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_clean_scrub_stamp":"2024-04-21T10:24:28.367927+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:20:12.106231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096569+0000","last_change":"2024-04-21T10:29:01.929293+0000","last_active":"2024-04-21T10:29:03.096569+0000","last_peered":"2024-04-21T10:29:03.096569+0000","last_clean":"2024-04-21T10:29:03.096569+0000","last_became_active":"2024-04-21T10:29:01.928942+0000","last_became_peered":"2024-04-21T10:29:01.928942+0000","last_unstale":"2024-04-21T10:29:03.096569+0000","last_undegraded":"2024-04-21T10:29:03.096569+0000","last_fullsized":"2024-04-21T10:29:03.096569+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:35.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.813224+0000","last_change":"2024-04-21T10:29:00.927521+0000","last_active":"2024-04-21T10:29:04.813224+0000","last_peered":"2024-04-21T10:29:04.813224+0000","last_clean":"2024-04-21T10:29:04.813224+0000","last_became_active":"2024-04-21T10:29:00.927202+0000","last_became_peered":"2024-04-21T10:29:00.927202+0000","last_unstale":"2024-04-21T10:29:04.813224+0000","last_undegraded":"2024-04-21T10:29:04.813224+0000","last_fullsized":"2024-04-21T10:29:04.813224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:11:32.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.804026+0000","last_change":"2024-04-21T10:29:00.930728+0000","last_active":"2024-04-21T10:29:03.804026+0000","last_peered":"2024-04-21T10:29:03.804026+0000","last_clean":"2024-04-21T10:29:03.804026+0000","last_became_active":"2024-04-21T10:29:00.930457+0000","last_became_peered":"2024-04-21T10:29:00.930457+0000","last_unstale":"2024-04-21T10:29:03.804026+0000","last_undegraded":"2024-04-21T10:29:03.804026+0000","last_fullsized":"2024-04-21T10:29:03.804026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:28.032954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097702+0000","last_change":"2024-04-21T10:29:01.932278+0000","last_active":"2024-04-21T10:29:03.097702+0000","last_peered":"2024-04-21T10:29:03.097702+0000","last_clean":"2024-04-21T10:29:03.097702+0000","last_became_active":"2024-04-21T10:29:01.931774+0000","last_became_peered":"2024-04-21T10:29:01.931774+0000","last_unstale":"2024-04-21T10:29:03.097702+0000","last_undegraded":"2024-04-21T10:29:03.097702+0000","last_fullsized":"2024-04-21T10:29:03.097702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:34:25.858475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.813481+0000","last_change":"2024-04-21T10:29:00.926071+0000","last_active":"2024-04-21T10:29:04.813481+0000","last_peered":"2024-04-21T10:29:04.813481+0000","last_clean":"2024-04-21T10:29:04.813481+0000","last_became_active":"2024-04-21T10:29:00.925607+0000","last_became_peered":"2024-04-21T10:29:00.925607+0000","last_unstale":"2024-04-21T10:29:04.813481+0000","last_undegraded":"2024-04-21T10:29:04.813481+0000","last_fullsized":"2024-04-21T10:29:04.813481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:26.350652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096783+0000","last_change":"2024-04-21T10:29:01.941365+0000","last_active":"2024-04-21T10:29:03.096783+0000","last_peered":"2024-04-21T10:29:03.096783+0000","last_clean":"2024-04-21T10:29:03.096783+0000","last_became_active":"2024-04-21T10:29:01.940102+0000","last_became_peered":"2024-04-21T10:29:01.940102+0000","last_unstale":"2024-04-21T10:29:03.096783+0000","last_undegraded":"2024-04-21T10:29:03.096783+0000","last_fullsized":"2024-04-21T10:29:03.096783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:21.976855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.803180+0000","last_change":"2024-04-21T10:29:00.924322+0000","last_active":"2024-04-21T10:29:03.803180+0000","last_peered":"2024-04-21T10:29:03.803180+0000","last_clean":"2024-04-21T10:29:03.803180+0000","last_became_active":"2024-04-21T10:29:00.924016+0000","last_became_peered":"2024-04-21T10:29:00.924016+0000","last_unstale":"2024-04-21T10:29:03.803180+0000","last_undegraded":"2024-04-21T10:29:03.803180+0000","last_fullsized":"2024-04-21T10:29:03.803180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:49:24.004951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932773+0000","last_change":"2024-04-21T10:29:01.946315+0000","last_active":"2024-04-21T10:29:02.932773+0000","last_peered":"2024-04-21T10:29:02.932773+0000","last_clean":"2024-04-21T10:29:02.932773+0000","last_became_active":"2024-04-21T10:29:01.945920+0000","last_became_peered":"2024-04-21T10:29:01.945920+0000","last_unstale":"2024-04-21T10:29:02.932773+0000","last_undegraded":"2024-04-21T10:29:02.932773+0000","last_fullsized":"2024-04-21T10:29:02.932773+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:10:52.361615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097006+0000","last_change":"2024-04-21T10:29:00.919086+0000","last_active":"2024-04-21T10:29:03.097006+0000","last_peered":"2024-04-21T10:29:03.097006+0000","last_clean":"2024-04-21T10:29:03.097006+0000","last_became_active":"2024-04-21T10:29:00.918727+0000","last_became_peered":"2024-04-21T10:29:00.918727+0000","last_unstale":"2024-04-21T10:29:03.097006+0000","last_undegraded":"2024-04-21T10:29:03.097006+0000","last_fullsized":"2024-04-21T10:29:03.097006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:53:27.209135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.934128+0000","last_change":"2024-04-21T10:29:01.944037+0000","last_active":"2024-04-21T10:29:02.934128+0000","last_peered":"2024-04-21T10:29:02.934128+0000","last_clean":"2024-04-21T10:29:02.934128+0000","last_became_active":"2024-04-21T10:29:01.943692+0000","last_became_peered":"2024-04-21T10:29:01.943692+0000","last_unstale":"2024-04-21T10:29:02.934128+0000","last_undegraded":"2024-04-21T10:29:02.934128+0000","last_fullsized":"2024-04-21T10:29:02.934128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:51.285535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.804286+0000","last_change":"2024-04-21T10:29:00.930937+0000","last_active":"2024-04-21T10:29:03.804286+0000","last_peered":"2024-04-21T10:29:03.804286+0000","last_clean":"2024-04-21T10:29:03.804286+0000","last_became_active":"2024-04-21T10:29:00.930348+0000","last_became_peered":"2024-04-21T10:29:00.930348+0000","last_unstale":"2024-04-21T10:29:03.804286+0000","last_undegraded":"2024-04-21T10:29:03.804286+0000","last_fullsized":"2024-04-21T10:29:03.804286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:43:33.801432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096986+0000","last_change":"2024-04-21T10:29:01.937116+0000","last_active":"2024-04-21T10:29:03.096986+0000","last_peered":"2024-04-21T10:29:03.096986+0000","last_clean":"2024-04-21T10:29:03.096986+0000","last_became_active":"2024-04-21T10:29:01.936756+0000","last_became_peered":"2024-04-21T10:29:01.936756+0000","last_unstale":"2024-04-21T10:29:03.096986+0000","last_undegraded":"2024-04-21T10:29:03.096986+0000","last_fullsized":"2024-04-21T10:29:03.096986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:02.732399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098255+0000","last_change":"2024-04-21T10:29:00.919355+0000","last_active":"2024-04-21T10:29:03.098255+0000","last_peered":"2024-04-21T10:29:03.098255+0000","last_clean":"2024-04-21T10:29:03.098255+0000","last_became_active":"2024-04-21T10:29:00.919095+0000","last_became_peered":"2024-04-21T10:29:00.919095+0000","last_unstale":"2024-04-21T10:29:03.098255+0000","last_undegraded":"2024-04-21T10:29:03.098255+0000","last_fullsized":"2024-04-21T10:29:03.098255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:07:34.821171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097755+0000","last_change":"2024-04-21T10:29:01.940677+0000","last_active":"2024-04-21T10:29:03.097755+0000","last_peered":"2024-04-21T10:29:03.097755+0000","last_clean":"2024-04-21T10:29:03.097755+0000","last_became_active":"2024-04-21T10:29:01.940296+0000","last_became_peered":"2024-04-21T10:29:01.940296+0000","last_unstale":"2024-04-21T10:29:03.097755+0000","last_undegraded":"2024-04-21T10:29:03.097755+0000","last_fullsized":"2024-04-21T10:29:03.097755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:24:52.662041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.806404+0000","last_change":"2024-04-21T10:29:00.924392+0000","last_active":"2024-04-21T10:29:03.806404+0000","last_peered":"2024-04-21T10:29:03.806404+0000","last_clean":"2024-04-21T10:29:03.806404+0000","last_became_active":"2024-04-21T10:29:00.924029+0000","last_became_peered":"2024-04-21T10:29:00.924029+0000","last_unstale":"2024-04-21T10:29:03.806404+0000","last_undegraded":"2024-04-21T10:29:03.806404+0000","last_fullsized":"2024-04-21T10:29:03.806404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:23:50.726978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.937954+0000","last_change":"2024-04-21T10:29:01.939687+0000","last_active":"2024-04-21T10:29:02.937954+0000","last_peered":"2024-04-21T10:29:02.937954+0000","last_clean":"2024-04-21T10:29:02.937954+0000","last_became_active":"2024-04-21T10:29:01.939343+0000","last_became_peered":"2024-04-21T10:29:01.939343+0000","last_unstale":"2024-04-21T10:29:02.937954+0000","last_undegraded":"2024-04-21T10:29:02.937954+0000","last_fullsized":"2024-04-21T10:29:02.937954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:50.481998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.816898+0000","last_change":"2024-04-21T10:29:00.923115+0000","last_active":"2024-04-21T10:29:04.816898+0000","last_peered":"2024-04-21T10:29:04.816898+0000","last_clean":"2024-04-21T10:29:04.816898+0000","last_became_active":"2024-04-21T10:29:00.922707+0000","last_became_peered":"2024-04-21T10:29:00.922707+0000","last_unstale":"2024-04-21T10:29:04.816898+0000","last_undegraded":"2024-04-21T10:29:04.816898+0000","last_fullsized":"2024-04-21T10:29:04.816898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:55:53.419904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.938105+0000","last_change":"2024-04-21T10:29:01.940666+0000","last_active":"2024-04-21T10:29:02.938105+0000","last_peered":"2024-04-21T10:29:02.938105+0000","last_clean":"2024-04-21T10:29:02.938105+0000","last_became_active":"2024-04-21T10:29:01.940331+0000","last_became_peered":"2024-04-21T10:29:01.940331+0000","last_unstale":"2024-04-21T10:29:02.938105+0000","last_undegraded":"2024-04-21T10:29:02.938105+0000","last_fullsized":"2024-04-21T10:29:02.938105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:52:41.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098230+0000","last_change":"2024-04-21T10:29:00.927873+0000","last_active":"2024-04-21T10:29:03.098230+0000","last_peered":"2024-04-21T10:29:03.098230+0000","last_clean":"2024-04-21T10:29:03.098230+0000","last_became_active":"2024-04-21T10:29:00.927403+0000","last_became_peered":"2024-04-21T10:29:00.927403+0000","last_unstale":"2024-04-21T10:29:03.098230+0000","last_undegraded":"2024-04-21T10:29:03.098230+0000","last_fullsized":"2024-04-21T10:29:03.098230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:05:05.276978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.937854+0000","last_change":"2024-04-21T10:29:01.944725+0000","last_active":"2024-04-21T10:29:02.937854+0000","last_peered":"2024-04-21T10:29:02.937854+0000","last_clean":"2024-04-21T10:29:02.937854+0000","last_became_active":"2024-04-21T10:29:01.944326+0000","last_became_peered":"2024-04-21T10:29:01.944326+0000","last_unstale":"2024-04-21T10:29:02.937854+0000","last_undegraded":"2024-04-21T10:29:02.937854+0000","last_fullsized":"2024-04-21T10:29:02.937854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:50.209064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096952+0000","last_change":"2024-04-21T10:29:00.924694+0000","last_active":"2024-04-21T10:29:03.096952+0000","last_peered":"2024-04-21T10:29:03.096952+0000","last_clean":"2024-04-21T10:29:03.096952+0000","last_became_active":"2024-04-21T10:29:00.924373+0000","last_became_peered":"2024-04-21T10:29:00.924373+0000","last_unstale":"2024-04-21T10:29:03.096952+0000","last_undegraded":"2024-04-21T10:29:03.096952+0000","last_fullsized":"2024-04-21T10:29:03.096952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:10:14.681630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.938194+0000","last_change":"2024-04-21T10:29:01.939277+0000","last_active":"2024-04-21T10:29:02.938194+0000","last_peered":"2024-04-21T10:29:02.938194+0000","last_clean":"2024-04-21T10:29:02.938194+0000","last_became_active":"2024-04-21T10:29:01.938844+0000","last_became_peered":"2024-04-21T10:29:01.938844+0000","last_unstale":"2024-04-21T10:29:02.938194+0000","last_undegraded":"2024-04-21T10:29:02.938194+0000","last_fullsized":"2024-04-21T10:29:02.938194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:38:14.553868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097181+0000","last_change":"2024-04-21T10:29:01.942654+0000","last_active":"2024-04-21T10:29:03.097181+0000","last_peered":"2024-04-21T10:29:03.097181+0000","last_clean":"2024-04-21T10:29:03.097181+0000","last_became_active":"2024-04-21T10:29:01.942324+0000","last_became_peered":"2024-04-21T10:29:01.942324+0000","last_unstale":"2024-04-21T10:29:03.097181+0000","last_undegraded":"2024-04-21T10:29:03.097181+0000","last_fullsized":"2024-04-21T10:29:03.097181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:19.027138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098173+0000","last_change":"2024-04-21T10:29:01.943088+0000","last_active":"2024-04-21T10:29:03.098173+0000","last_peered":"2024-04-21T10:29:03.098173+0000","last_clean":"2024-04-21T10:29:03.098173+0000","last_became_active":"2024-04-21T10:29:01.942542+0000","last_became_peered":"2024-04-21T10:29:01.942542+0000","last_unstale":"2024-04-21T10:29:03.098173+0000","last_undegraded":"2024-04-21T10:29:03.098173+0000","last_fullsized":"2024-04-21T10:29:03.098173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:50:55.283349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098011+0000","last_change":"2024-04-21T10:29:01.944405+0000","last_active":"2024-04-21T10:29:03.098011+0000","last_peered":"2024-04-21T10:29:03.098011+0000","last_clean":"2024-04-21T10:29:03.098011+0000","last_became_active":"2024-04-21T10:29:01.944085+0000","last_became_peered":"2024-04-21T10:29:01.944085+0000","last_unstale":"2024-04-21T10:29:03.098011+0000","last_undegraded":"2024-04-21T10:29:03.098011+0000","last_fullsized":"2024-04-21T10:29:03.098011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:13.299859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932624+0000","last_change":"2024-04-21T10:29:01.941054+0000","last_active":"2024-04-21T10:29:02.932624+0000","last_peered":"2024-04-21T10:29:02.932624+0000","last_clean":"2024-04-21T10:29:02.932624+0000","last_became_active":"2024-04-21T10:29:01.940243+0000","last_became_peered":"2024-04-21T10:29:01.940243+0000","last_unstale":"2024-04-21T10:29:02.932624+0000","last_undegraded":"2024-04-21T10:29:02.932624+0000","last_fullsized":"2024-04-21T10:29:02.932624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:31:04.222026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097638+0000","last_change":"2024-04-21T10:29:01.934315+0000","last_active":"2024-04-21T10:29:03.097638+0000","last_peered":"2024-04-21T10:29:03.097638+0000","last_clean":"2024-04-21T10:29:03.097638+0000","last_became_active":"2024-04-21T10:29:01.933996+0000","last_became_peered":"2024-04-21T10:29:01.933996+0000","last_unstale":"2024-04-21T10:29:03.097638+0000","last_undegraded":"2024-04-21T10:29:03.097638+0000","last_fullsized":"2024-04-21T10:29:03.097638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:07.374591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097903+0000","last_change":"2024-04-21T10:29:01.947268+0000","last_active":"2024-04-21T10:29:03.097903+0000","last_peered":"2024-04-21T10:29:03.097903+0000","last_clean":"2024-04-21T10:29:03.097903+0000","last_became_active":"2024-04-21T10:29:01.946921+0000","last_became_peered":"2024-04-21T10:29:01.946921+0000","last_unstale":"2024-04-21T10:29:03.097903+0000","last_undegraded":"2024-04-21T10:29:03.097903+0000","last_fullsized":"2024-04-21T10:29:03.097903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:50:43.784920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098783+0000","last_change":"2024-04-21T10:29:01.949969+0000","last_active":"2024-04-21T10:29:03.098783+0000","last_peered":"2024-04-21T10:29:03.098783+0000","last_clean":"2024-04-21T10:29:03.098783+0000","last_became_active":"2024-04-21T10:29:01.949487+0000","last_became_peered":"2024-04-21T10:29:01.949487+0000","last_unstale":"2024-04-21T10:29:03.098783+0000","last_undegraded":"2024-04-21T10:29:03.098783+0000","last_fullsized":"2024-04-21T10:29:03.098783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:26:59.620209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096650+0000","last_change":"2024-04-21T10:29:01.937904+0000","last_active":"2024-04-21T10:29:03.096650+0000","last_peered":"2024-04-21T10:29:03.096650+0000","last_clean":"2024-04-21T10:29:03.096650+0000","last_became_active":"2024-04-21T10:29:01.937504+0000","last_became_peered":"2024-04-21T10:29:01.937504+0000","last_unstale":"2024-04-21T10:29:03.096650+0000","last_undegraded":"2024-04-21T10:29:03.096650+0000","last_fullsized":"2024-04-21T10:29:03.096650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:58:24.245118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097183+0000","last_change":"2024-04-21T10:29:01.934320+0000","last_active":"2024-04-21T10:29:03.097183+0000","last_peered":"2024-04-21T10:29:03.097183+0000","last_clean":"2024-04-21T10:29:03.097183+0000","last_became_active":"2024-04-21T10:29:01.934015+0000","last_became_peered":"2024-04-21T10:29:01.934015+0000","last_unstale":"2024-04-21T10:29:03.097183+0000","last_undegraded":"2024-04-21T10:29:03.097183+0000","last_fullsized":"2024-04-21T10:29:03.097183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:31:39.700427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097341+0000","last_change":"2024-04-21T10:29:01.936880+0000","last_active":"2024-04-21T10:29:03.097341+0000","last_peered":"2024-04-21T10:29:03.097341+0000","last_clean":"2024-04-21T10:29:03.097341+0000","last_became_active":"2024-04-21T10:29:01.936555+0000","last_became_peered":"2024-04-21T10:29:01.936555+0000","last_unstale":"2024-04-21T10:29:03.097341+0000","last_undegraded":"2024-04-21T10:29:03.097341+0000","last_fullsized":"2024-04-21T10:29:03.097341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:32.831606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097772+0000","last_change":"2024-04-21T10:29:01.943238+0000","last_active":"2024-04-21T10:29:03.097772+0000","last_peered":"2024-04-21T10:29:03.097772+0000","last_clean":"2024-04-21T10:29:03.097772+0000","last_became_active":"2024-04-21T10:29:01.942841+0000","last_became_peered":"2024-04-21T10:29:01.942841+0000","last_unstale":"2024-04-21T10:29:03.097772+0000","last_undegraded":"2024-04-21T10:29:03.097772+0000","last_fullsized":"2024-04-21T10:29:03.097772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:59:48.551524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097387+0000","last_change":"2024-04-21T10:29:01.933712+0000","last_active":"2024-04-21T10:29:03.097387+0000","last_peered":"2024-04-21T10:29:03.097387+0000","last_clean":"2024-04-21T10:29:03.097387+0000","last_became_active":"2024-04-21T10:29:01.933399+0000","last_became_peered":"2024-04-21T10:29:01.933399+0000","last_unstale":"2024-04-21T10:29:03.097387+0000","last_undegraded":"2024-04-21T10:29:03.097387+0000","last_fullsized":"2024-04-21T10:29:03.097387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:34:16.346403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097336+0000","last_change":"2024-04-21T10:29:01.947225+0000","last_active":"2024-04-21T10:29:03.097336+0000","last_peered":"2024-04-21T10:29:03.097336+0000","last_clean":"2024-04-21T10:29:03.097336+0000","last_became_active":"2024-04-21T10:29:01.946890+0000","last_became_peered":"2024-04-21T10:29:01.946890+0000","last_unstale":"2024-04-21T10:29:03.097336+0000","last_undegraded":"2024-04-21T10:29:03.097336+0000","last_fullsized":"2024-04-21T10:29:03.097336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:19:00.960767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24896,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":33,"num_read_kb":45,"num_write":130,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":72312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":446648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1046615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743448,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":446884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005085,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":446804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266721,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1037638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":446156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":439108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":447544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":468311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":446350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":447101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32696,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712456,"statfs":{"total":95995035648,"available":95961554944,"internally_reserved":0,"allocated":1556480,"data_stored":1037910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":39543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T10:29:08.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T10:29:08.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.808+0000 7fd556d0d640 1 -- 172.21.15.26:0/2114201540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd550103bf0 msgr2=0x7fd55010a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.808+0000 7fd556d0d640 1 --2- 172.21.15.26:0/2114201540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd550103bf0 0x7fd55010a480 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd540009930 tx=0x7fd54002bc30 comp rx=0 tx=0).stop 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.808+0000 7fd556d0d640 1 -- 172.21.15.26:0/2114201540 shutdown_connections 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.808+0000 7fd556d0d640 1 --2- 172.21.15.26:0/2114201540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd550103bf0 0x7fd55010a480 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.808+0000 7fd556d0d640 1 --2- 172.21.15.26:0/2114201540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd550103230 0x7fd5501036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.808+0000 7fd556d0d640 1 --2- 172.21.15.26:0/2114201540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd550100260 0x7fd550100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.808+0000 7fd556d0d640 1 -- 172.21.15.26:0/2114201540 >> 172.21.15.26:0/2114201540 conn(0x7fd5500fbb90 msgr2=0x7fd5500fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.808+0000 7fd556d0d640 1 -- 172.21.15.26:0/2114201540 shutdown_connections 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.808+0000 7fd556d0d640 1 -- 172.21.15.26:0/2114201540 wait complete. 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.808+0000 7fd556d0d640 1 Processor -- start 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.808+0000 7fd556d0d640 1 -- start start 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd556d0d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd550100260 0x7fd5501981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd556d0d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd550103230 0x7fd550198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd556d0d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd550103bf0 0x7fd55019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd556d0d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd55010db60 con 0x7fd550100260 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd556d0d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd55010d9e0 con 0x7fd550103230 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd556d0d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd55010dce0 con 0x7fd550103bf0 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd554aa9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd550100260 0x7fd5501981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd554aa9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd550100260 0x7fd5501981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58952/0 (socket says 172.21.15.26:58952) 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd554aa9640 1 -- 172.21.15.26:0/512090938 learned_addr learned my addr 172.21.15.26:0/512090938 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd554aa9640 1 -- 172.21.15.26:0/512090938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd550103bf0 msgr2=0x7fd55019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd554aa9640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd550103bf0 0x7fd55019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd554aa9640 1 -- 172.21.15.26:0/512090938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd550103230 msgr2=0x7fd550198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd54ffff640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd550103230 0x7fd550198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd554aa9640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd550103230 0x7fd550198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd554aa9640 1 -- 172.21.15.26:0/512090938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55019fe00 con 0x7fd550100260 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd554aa9640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd550100260 0x7fd5501981e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b9c0 tx=0x7fd54800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd54dffb640 1 -- 172.21.15.26:0/512090938 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548019070 con 0x7fd550100260 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd556d0d640 1 -- 172.21.15.26:0/512090938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5501a00f0 con 0x7fd550100260 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd54dffb640 1 -- 172.21.15.26:0/512090938 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd54800cba0 con 0x7fd550100260 2024-04-21T10:29:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.809+0000 7fd54dffb640 1 -- 172.21.15.26:0/512090938 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5480125a0 con 0x7fd550100260 2024-04-21T10:29:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.810+0000 7fd556d0d640 1 -- 172.21.15.26:0/512090938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5501a0710 con 0x7fd550100260 2024-04-21T10:29:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.810+0000 7fd556d0d640 1 -- 172.21.15.26:0/512090938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd550109360 con 0x7fd550100260 2024-04-21T10:29:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.811+0000 7fd54dffb640 1 -- 172.21.15.26:0/512090938 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd54801c070 con 0x7fd550100260 2024-04-21T10:29:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.812+0000 7fd54dffb640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd53c0a3c40 0x7fd53c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:08.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.812+0000 7fd54ffff640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd53c0a3c40 0x7fd53c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:08.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.812+0000 7fd54dffb640 1 -- 172.21.15.26:0/512090938 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd5480cf8e0 con 0x7fd550100260 2024-04-21T10:29:08.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.813+0000 7fd54ffff640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd53c0a3c40 0x7fd53c0a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd538005fd0 tx=0x7fd538005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:08.817+0000 7fd54dffb640 1 -- 172.21.15.26:0/512090938 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd54809c420 con 0x7fd550100260 2024-04-21T10:29:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.003+0000 7fd556d0d640 1 -- 172.21.15.26:0/512090938 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd550062390 con 0x7fd53c0a3c40 2024-04-21T10:29:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.010+0000 7fd54dffb640 1 -- 172.21.15.26:0/512090938 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258757 (secure 0 0 0) 0x7fd550062390 con 0x7fd53c0a3c40 2024-04-21T10:29:09.013 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:29:09.014 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-21T10:29:09.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.013+0000 7fd556d0d640 1 -- 172.21.15.26:0/512090938 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd53c0a3c40 msgr2=0x7fd53c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:09.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.013+0000 7fd556d0d640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd53c0a3c40 0x7fd53c0a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd538005fd0 tx=0x7fd538005850 comp rx=0 tx=0).stop 2024-04-21T10:29:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.013+0000 7fd556d0d640 1 -- 172.21.15.26:0/512090938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd550100260 msgr2=0x7fd5501981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.013+0000 7fd556d0d640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd550100260 0x7fd5501981e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b9c0 tx=0x7fd54800be80 comp rx=0 tx=0).stop 2024-04-21T10:29:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.013+0000 7fd556d0d640 1 -- 172.21.15.26:0/512090938 shutdown_connections 2024-04-21T10:29:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.013+0000 7fd556d0d640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd53c0a3c40 0x7fd53c0a6100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.013+0000 7fd556d0d640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd550103bf0 0x7fd55019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.013+0000 7fd556d0d640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd550103230 0x7fd550198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.013+0000 7fd556d0d640 1 --2- 172.21.15.26:0/512090938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd550100260 0x7fd5501981e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.013+0000 7fd556d0d640 1 -- 172.21.15.26:0/512090938 >> 172.21.15.26:0/512090938 conn(0x7fd5500fbb90 msgr2=0x7fd550106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.014+0000 7fd556d0d640 1 -- 172.21.15.26:0/512090938 shutdown_connections 2024-04-21T10:29:09.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:09.014+0000 7fd556d0d640 1 -- 172.21.15.26:0/512090938 wait complete. 2024-04-21T10:29:09.025 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-04-21T10:29:08.179286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":615381,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":169,"num_read_kb":164,"num_write":312,"num_write_kb":2999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":253,"ondisk_log_size":253,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362092,"kb_used_data":12928,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579732,"statfs":{"total":1151940427776,"available":1151569645568,"internally_reserved":0,"allocated":13238272,"data_stored":7157069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25013,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":33,"num_read_kb":45,"num_write":131,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.269046"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098382+0000","last_change":"2024-04-21T10:29:01.940996+0000","last_active":"2024-04-21T10:29:03.098382+0000","last_peered":"2024-04-21T10:29:03.098382+0000","last_clean":"2024-04-21T10:29:03.098382+0000","last_became_active":"2024-04-21T10:29:01.940662+0000","last_became_peered":"2024-04-21T10:29:01.940662+0000","last_unstale":"2024-04-21T10:29:03.098382+0000","last_undegraded":"2024-04-21T10:29:03.098382+0000","last_fullsized":"2024-04-21T10:29:03.098382+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:16:01.434895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932733+0000","last_change":"2024-04-21T10:29:01.939238+0000","last_active":"2024-04-21T10:29:02.932733+0000","last_peered":"2024-04-21T10:29:02.932733+0000","last_clean":"2024-04-21T10:29:02.932733+0000","last_became_active":"2024-04-21T10:29:01.938907+0000","last_became_peered":"2024-04-21T10:29:01.938907+0000","last_unstale":"2024-04-21T10:29:02.932733+0000","last_undegraded":"2024-04-21T10:29:02.932733+0000","last_fullsized":"2024-04-21T10:29:02.932733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:09.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932734+0000","last_change":"2024-04-21T10:29:01.932338+0000","last_active":"2024-04-21T10:29:02.932734+0000","last_peered":"2024-04-21T10:29:02.932734+0000","last_clean":"2024-04-21T10:29:02.932734+0000","last_became_active":"2024-04-21T10:29:01.931916+0000","last_became_peered":"2024-04-21T10:29:01.931916+0000","last_unstale":"2024-04-21T10:29:02.932734+0000","last_undegraded":"2024-04-21T10:29:02.932734+0000","last_fullsized":"2024-04-21T10:29:02.932734+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:52.859695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097400+0000","last_change":"2024-04-21T10:29:01.945572+0000","last_active":"2024-04-21T10:29:03.097400+0000","last_peered":"2024-04-21T10:29:03.097400+0000","last_clean":"2024-04-21T10:29:03.097400+0000","last_became_active":"2024-04-21T10:29:01.940114+0000","last_became_peered":"2024-04-21T10:29:01.940114+0000","last_unstale":"2024-04-21T10:29:03.097400+0000","last_undegraded":"2024-04-21T10:29:03.097400+0000","last_fullsized":"2024-04-21T10:29:03.097400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:21.935573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.934195+0000","last_change":"2024-04-21T10:29:01.946021+0000","last_active":"2024-04-21T10:29:02.934195+0000","last_peered":"2024-04-21T10:29:02.934195+0000","last_clean":"2024-04-21T10:29:02.934195+0000","last_became_active":"2024-04-21T10:29:01.945637+0000","last_became_peered":"2024-04-21T10:29:01.945637+0000","last_unstale":"2024-04-21T10:29:02.934195+0000","last_undegraded":"2024-04-21T10:29:02.934195+0000","last_fullsized":"2024-04-21T10:29:02.934195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:20.840494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097299+0000","last_change":"2024-04-21T10:29:01.935402+0000","last_active":"2024-04-21T10:29:03.097299+0000","last_peered":"2024-04-21T10:29:03.097299+0000","last_clean":"2024-04-21T10:29:03.097299+0000","last_became_active":"2024-04-21T10:29:01.934994+0000","last_became_peered":"2024-04-21T10:29:01.934994+0000","last_unstale":"2024-04-21T10:29:03.097299+0000","last_undegraded":"2024-04-21T10:29:03.097299+0000","last_fullsized":"2024-04-21T10:29:03.097299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:18.320530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.937751+0000","last_change":"2024-04-21T10:29:01.943185+0000","last_active":"2024-04-21T10:29:02.937751+0000","last_peered":"2024-04-21T10:29:02.937751+0000","last_clean":"2024-04-21T10:29:02.937751+0000","last_became_active":"2024-04-21T10:29:01.942805+0000","last_became_peered":"2024-04-21T10:29:01.942805+0000","last_unstale":"2024-04-21T10:29:02.937751+0000","last_undegraded":"2024-04-21T10:29:02.937751+0000","last_fullsized":"2024-04-21T10:29:02.937751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:06.776472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096682+0000","last_change":"2024-04-21T10:29:01.940995+0000","last_active":"2024-04-21T10:29:03.096682+0000","last_peered":"2024-04-21T10:29:03.096682+0000","last_clean":"2024-04-21T10:29:03.096682+0000","last_became_active":"2024-04-21T10:29:01.939173+0000","last_became_peered":"2024-04-21T10:29:01.939173+0000","last_unstale":"2024-04-21T10:29:03.096682+0000","last_undegraded":"2024-04-21T10:29:03.096682+0000","last_fullsized":"2024-04-21T10:29:03.096682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.740147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098178+0000","last_change":"2024-04-21T10:29:01.944284+0000","last_active":"2024-04-21T10:29:03.098178+0000","last_peered":"2024-04-21T10:29:03.098178+0000","last_clean":"2024-04-21T10:29:03.098178+0000","last_became_active":"2024-04-21T10:29:01.943965+0000","last_became_peered":"2024-04-21T10:29:01.943965+0000","last_unstale":"2024-04-21T10:29:03.098178+0000","last_undegraded":"2024-04-21T10:29:03.098178+0000","last_fullsized":"2024-04-21T10:29:03.098178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:08:54.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098288+0000","last_change":"2024-04-21T10:29:01.936250+0000","last_active":"2024-04-21T10:29:03.098288+0000","last_peered":"2024-04-21T10:29:03.098288+0000","last_clean":"2024-04-21T10:29:03.098288+0000","last_became_active":"2024-04-21T10:29:01.935904+0000","last_became_peered":"2024-04-21T10:29:01.935904+0000","last_unstale":"2024-04-21T10:29:03.098288+0000","last_undegraded":"2024-04-21T10:29:03.098288+0000","last_fullsized":"2024-04-21T10:29:03.098288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:35.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.938138+0000","last_change":"2024-04-21T10:29:01.946550+0000","last_active":"2024-04-21T10:29:02.938138+0000","last_peered":"2024-04-21T10:29:02.938138+0000","last_clean":"2024-04-21T10:29:02.938138+0000","last_became_active":"2024-04-21T10:29:01.946170+0000","last_became_peered":"2024-04-21T10:29:01.946170+0000","last_unstale":"2024-04-21T10:29:02.938138+0000","last_undegraded":"2024-04-21T10:29:02.938138+0000","last_fullsized":"2024-04-21T10:29:02.938138+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:25:47.671821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097217+0000","last_change":"2024-04-21T10:29:01.946036+0000","last_active":"2024-04-21T10:29:03.097217+0000","last_peered":"2024-04-21T10:29:03.097217+0000","last_clean":"2024-04-21T10:29:03.097217+0000","last_became_active":"2024-04-21T10:29:01.940705+0000","last_became_peered":"2024-04-21T10:29:01.940705+0000","last_unstale":"2024-04-21T10:29:03.097217+0000","last_undegraded":"2024-04-21T10:29:03.097217+0000","last_fullsized":"2024-04-21T10:29:03.097217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:11:33.196059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097218+0000","last_change":"2024-04-21T10:29:01.940438+0000","last_active":"2024-04-21T10:29:03.097218+0000","last_peered":"2024-04-21T10:29:03.097218+0000","last_clean":"2024-04-21T10:29:03.097218+0000","last_became_active":"2024-04-21T10:29:01.940085+0000","last_became_peered":"2024-04-21T10:29:01.940085+0000","last_unstale":"2024-04-21T10:29:03.097218+0000","last_undegraded":"2024-04-21T10:29:03.097218+0000","last_fullsized":"2024-04-21T10:29:03.097218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:22:47.828858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932399+0000","last_change":"2024-04-21T10:29:01.941667+0000","last_active":"2024-04-21T10:29:02.932399+0000","last_peered":"2024-04-21T10:29:02.932399+0000","last_clean":"2024-04-21T10:29:02.932399+0000","last_became_active":"2024-04-21T10:29:01.940745+0000","last_became_peered":"2024-04-21T10:29:01.940745+0000","last_unstale":"2024-04-21T10:29:02.932399+0000","last_undegraded":"2024-04-21T10:29:02.932399+0000","last_fullsized":"2024-04-21T10:29:02.932399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:45:51.825780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932401+0000","last_change":"2024-04-21T10:29:01.947771+0000","last_active":"2024-04-21T10:29:02.932401+0000","last_peered":"2024-04-21T10:29:02.932401+0000","last_clean":"2024-04-21T10:29:02.932401+0000","last_became_active":"2024-04-21T10:29:01.946818+0000","last_became_peered":"2024-04-21T10:29:01.946818+0000","last_unstale":"2024-04-21T10:29:02.932401+0000","last_undegraded":"2024-04-21T10:29:02.932401+0000","last_fullsized":"2024-04-21T10:29:02.932401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:18:42.159152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096886+0000","last_change":"2024-04-21T10:29:01.942499+0000","last_active":"2024-04-21T10:29:03.096886+0000","last_peered":"2024-04-21T10:29:03.096886+0000","last_clean":"2024-04-21T10:29:03.096886+0000","last_became_active":"2024-04-21T10:29:01.942136+0000","last_became_peered":"2024-04-21T10:29:01.942136+0000","last_unstale":"2024-04-21T10:29:03.096886+0000","last_undegraded":"2024-04-21T10:29:03.096886+0000","last_fullsized":"2024-04-21T10:29:03.096886+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:34:11.822230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098546+0000","last_change":"2024-04-21T10:29:01.948822+0000","last_active":"2024-04-21T10:29:03.098546+0000","last_peered":"2024-04-21T10:29:03.098546+0000","last_clean":"2024-04-21T10:29:03.098546+0000","last_became_active":"2024-04-21T10:29:01.948492+0000","last_became_peered":"2024-04-21T10:29:01.948492+0000","last_unstale":"2024-04-21T10:29:03.098546+0000","last_undegraded":"2024-04-21T10:29:03.098546+0000","last_fullsized":"2024-04-21T10:29:03.098546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:06.992585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096820+0000","last_change":"2024-04-21T10:29:01.931265+0000","last_active":"2024-04-21T10:29:03.096820+0000","last_peered":"2024-04-21T10:29:03.096820+0000","last_clean":"2024-04-21T10:29:03.096820+0000","last_became_active":"2024-04-21T10:29:01.930877+0000","last_became_peered":"2024-04-21T10:29:01.930877+0000","last_unstale":"2024-04-21T10:29:03.096820+0000","last_undegraded":"2024-04-21T10:29:03.096820+0000","last_fullsized":"2024-04-21T10:29:03.096820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:49:00.951226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098552+0000","last_change":"2024-04-21T10:29:01.941718+0000","last_active":"2024-04-21T10:29:03.098552+0000","last_peered":"2024-04-21T10:29:03.098552+0000","last_clean":"2024-04-21T10:29:03.098552+0000","last_became_active":"2024-04-21T10:29:01.941386+0000","last_became_peered":"2024-04-21T10:29:01.941386+0000","last_unstale":"2024-04-21T10:29:03.098552+0000","last_undegraded":"2024-04-21T10:29:03.098552+0000","last_fullsized":"2024-04-21T10:29:03.098552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:51:29.847961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.814006+0000","last_change":"2024-04-21T10:29:00.923670+0000","last_active":"2024-04-21T10:29:04.814006+0000","last_peered":"2024-04-21T10:29:04.814006+0000","last_clean":"2024-04-21T10:29:04.814006+0000","last_became_active":"2024-04-21T10:29:00.923100+0000","last_became_peered":"2024-04-21T10:29:00.923100+0000","last_unstale":"2024-04-21T10:29:04.814006+0000","last_undegraded":"2024-04-21T10:29:04.814006+0000","last_fullsized":"2024-04-21T10:29:04.814006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:45.692961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096884+0000","last_change":"2024-04-21T10:29:01.938131+0000","last_active":"2024-04-21T10:29:03.096884+0000","last_peered":"2024-04-21T10:29:03.096884+0000","last_clean":"2024-04-21T10:29:03.096884+0000","last_became_active":"2024-04-21T10:29:01.937752+0000","last_became_peered":"2024-04-21T10:29:01.937752+0000","last_unstale":"2024-04-21T10:29:03.096884+0000","last_undegraded":"2024-04-21T10:29:03.096884+0000","last_fullsized":"2024-04-21T10:29:03.096884+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:06:49.383554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096995+0000","last_change":"2024-04-21T10:29:00.929877+0000","last_active":"2024-04-21T10:29:03.096995+0000","last_peered":"2024-04-21T10:29:03.096995+0000","last_clean":"2024-04-21T10:29:03.096995+0000","last_became_active":"2024-04-21T10:29:00.929564+0000","last_became_peered":"2024-04-21T10:29:00.929564+0000","last_unstale":"2024-04-21T10:29:03.096995+0000","last_undegraded":"2024-04-21T10:29:03.096995+0000","last_fullsized":"2024-04-21T10:29:03.096995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:35.617533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932928+0000","last_change":"2024-04-21T10:29:01.942782+0000","last_active":"2024-04-21T10:29:02.932928+0000","last_peered":"2024-04-21T10:29:02.932928+0000","last_clean":"2024-04-21T10:29:02.932928+0000","last_became_active":"2024-04-21T10:29:01.940501+0000","last_became_peered":"2024-04-21T10:29:01.940501+0000","last_unstale":"2024-04-21T10:29:02.932928+0000","last_undegraded":"2024-04-21T10:29:02.932928+0000","last_fullsized":"2024-04-21T10:29:02.932928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:04:03.431016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.814189+0000","last_change":"2024-04-21T10:29:00.924395+0000","last_active":"2024-04-21T10:29:04.814189+0000","last_peered":"2024-04-21T10:29:04.814189+0000","last_clean":"2024-04-21T10:29:04.814189+0000","last_became_active":"2024-04-21T10:29:00.924045+0000","last_became_peered":"2024-04-21T10:29:00.924045+0000","last_unstale":"2024-04-21T10:29:04.814189+0000","last_undegraded":"2024-04-21T10:29:04.814189+0000","last_fullsized":"2024-04-21T10:29:04.814189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:08:53.922213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097980+0000","last_change":"2024-04-21T10:29:01.934508+0000","last_active":"2024-04-21T10:29:03.097980+0000","last_peered":"2024-04-21T10:29:03.097980+0000","last_clean":"2024-04-21T10:29:03.097980+0000","last_became_active":"2024-04-21T10:29:01.934051+0000","last_became_peered":"2024-04-21T10:29:01.934051+0000","last_unstale":"2024-04-21T10:29:03.097980+0000","last_undegraded":"2024-04-21T10:29:03.097980+0000","last_fullsized":"2024-04-21T10:29:03.097980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:49:27.686603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.934347+0000","last_change":"2024-04-21T10:29:00.925850+0000","last_active":"2024-04-21T10:29:02.934347+0000","last_peered":"2024-04-21T10:29:02.934347+0000","last_clean":"2024-04-21T10:29:02.934347+0000","last_became_active":"2024-04-21T10:29:00.925521+0000","last_became_peered":"2024-04-21T10:29:00.925521+0000","last_unstale":"2024-04-21T10:29:02.934347+0000","last_undegraded":"2024-04-21T10:29:02.934347+0000","last_fullsized":"2024-04-21T10:29:02.934347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:36:55.403413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098269+0000","last_change":"2024-04-21T10:29:01.930046+0000","last_active":"2024-04-21T10:29:03.098269+0000","last_peered":"2024-04-21T10:29:03.098269+0000","last_clean":"2024-04-21T10:29:03.098269+0000","last_became_active":"2024-04-21T10:29:01.929736+0000","last_became_peered":"2024-04-21T10:29:01.929736+0000","last_unstale":"2024-04-21T10:29:03.098269+0000","last_undegraded":"2024-04-21T10:29:03.098269+0000","last_fullsized":"2024-04-21T10:29:03.098269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:11:10.833267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:05.816158+0000","last_change":"2024-04-21T10:29:00.927936+0000","last_active":"2024-04-21T10:29:05.816158+0000","last_peered":"2024-04-21T10:29:05.816158+0000","last_clean":"2024-04-21T10:29:05.816158+0000","last_became_active":"2024-04-21T10:29:00.927352+0000","last_became_peered":"2024-04-21T10:29:00.927352+0000","last_unstale":"2024-04-21T10:29:05.816158+0000","last_undegraded":"2024-04-21T10:29:05.816158+0000","last_fullsized":"2024-04-21T10:29:05.816158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:14:10.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.934244+0000","last_change":"2024-04-21T10:29:01.943389+0000","last_active":"2024-04-21T10:29:02.934244+0000","last_peered":"2024-04-21T10:29:02.934244+0000","last_clean":"2024-04-21T10:29:02.934244+0000","last_became_active":"2024-04-21T10:29:01.940386+0000","last_became_peered":"2024-04-21T10:29:01.940386+0000","last_unstale":"2024-04-21T10:29:02.934244+0000","last_undegraded":"2024-04-21T10:29:02.934244+0000","last_fullsized":"2024-04-21T10:29:02.934244+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:11:54.894785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.813157+0000","last_change":"2024-04-21T10:29:00.925945+0000","last_active":"2024-04-21T10:29:04.813157+0000","last_peered":"2024-04-21T10:29:04.813157+0000","last_clean":"2024-04-21T10:29:04.813157+0000","last_became_active":"2024-04-21T10:29:00.925641+0000","last_became_peered":"2024-04-21T10:29:00.925641+0000","last_unstale":"2024-04-21T10:29:04.813157+0000","last_undegraded":"2024-04-21T10:29:04.813157+0000","last_fullsized":"2024-04-21T10:29:04.813157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:44:18.204033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097636+0000","last_change":"2024-04-21T10:29:01.918627+0000","last_active":"2024-04-21T10:29:03.097636+0000","last_peered":"2024-04-21T10:29:03.097636+0000","last_clean":"2024-04-21T10:29:03.097636+0000","last_became_active":"2024-04-21T10:29:01.917993+0000","last_became_peered":"2024-04-21T10:29:01.917993+0000","last_unstale":"2024-04-21T10:29:03.097636+0000","last_undegraded":"2024-04-21T10:29:03.097636+0000","last_fullsized":"2024-04-21T10:29:03.097636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:58.891918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.813855+0000","last_change":"2024-04-21T10:29:00.923613+0000","last_active":"2024-04-21T10:29:04.813855+0000","last_peered":"2024-04-21T10:29:04.813855+0000","last_clean":"2024-04-21T10:29:04.813855+0000","last_became_active":"2024-04-21T10:29:00.923001+0000","last_became_peered":"2024-04-21T10:29:00.923001+0000","last_unstale":"2024-04-21T10:29:04.813855+0000","last_undegraded":"2024-04-21T10:29:04.813855+0000","last_fullsized":"2024-04-21T10:29:04.813855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:44:53.593903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096485+0000","last_change":"2024-04-21T10:29:01.934974+0000","last_active":"2024-04-21T10:29:03.096485+0000","last_peered":"2024-04-21T10:29:03.096485+0000","last_clean":"2024-04-21T10:29:03.096485+0000","last_became_active":"2024-04-21T10:29:01.932443+0000","last_became_peered":"2024-04-21T10:29:01.932443+0000","last_unstale":"2024-04-21T10:29:03.096485+0000","last_undegraded":"2024-04-21T10:29:03.096485+0000","last_fullsized":"2024-04-21T10:29:03.096485+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:24:47.190142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097123+0000","last_change":"2024-04-21T10:29:00.928179+0000","last_active":"2024-04-21T10:29:03.097123+0000","last_peered":"2024-04-21T10:29:03.097123+0000","last_clean":"2024-04-21T10:29:03.097123+0000","last_became_active":"2024-04-21T10:29:00.927588+0000","last_became_peered":"2024-04-21T10:29:00.927588+0000","last_unstale":"2024-04-21T10:29:03.097123+0000","last_undegraded":"2024-04-21T10:29:03.097123+0000","last_fullsized":"2024-04-21T10:29:03.097123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:04:41.081424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097143+0000","last_change":"2024-04-21T10:29:01.944421+0000","last_active":"2024-04-21T10:29:03.097143+0000","last_peered":"2024-04-21T10:29:03.097143+0000","last_clean":"2024-04-21T10:29:03.097143+0000","last_became_active":"2024-04-21T10:29:01.943572+0000","last_became_peered":"2024-04-21T10:29:01.943572+0000","last_unstale":"2024-04-21T10:29:03.097143+0000","last_undegraded":"2024-04-21T10:29:03.097143+0000","last_fullsized":"2024-04-21T10:29:03.097143+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:34:01.690600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:05.814317+0000","last_change":"2024-04-21T10:29:00.924406+0000","last_active":"2024-04-21T10:29:05.814317+0000","last_peered":"2024-04-21T10:29:05.814317+0000","last_clean":"2024-04-21T10:29:05.814317+0000","last_became_active":"2024-04-21T10:29:00.924094+0000","last_became_peered":"2024-04-21T10:29:00.924094+0000","last_unstale":"2024-04-21T10:29:05.814317+0000","last_undegraded":"2024-04-21T10:29:05.814317+0000","last_fullsized":"2024-04-21T10:29:05.814317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:44.673751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098275+0000","last_change":"2024-04-21T10:29:01.933048+0000","last_active":"2024-04-21T10:29:03.098275+0000","last_peered":"2024-04-21T10:29:03.098275+0000","last_clean":"2024-04-21T10:29:03.098275+0000","last_became_active":"2024-04-21T10:29:01.932659+0000","last_became_peered":"2024-04-21T10:29:01.932659+0000","last_unstale":"2024-04-21T10:29:03.098275+0000","last_undegraded":"2024-04-21T10:29:03.098275+0000","last_fullsized":"2024-04-21T10:29:03.098275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:22:13.712112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.806580+0000","last_change":"2024-04-21T10:29:00.923020+0000","last_active":"2024-04-21T10:29:03.806580+0000","last_peered":"2024-04-21T10:29:03.806580+0000","last_clean":"2024-04-21T10:29:03.806580+0000","last_became_active":"2024-04-21T10:29:00.922678+0000","last_became_peered":"2024-04-21T10:29:00.922678+0000","last_unstale":"2024-04-21T10:29:03.806580+0000","last_undegraded":"2024-04-21T10:29:03.806580+0000","last_fullsized":"2024-04-21T10:29:03.806580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:13:34.698082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932938+0000","last_change":"2024-04-21T10:29:01.947833+0000","last_active":"2024-04-21T10:29:02.932938+0000","last_peered":"2024-04-21T10:29:02.932938+0000","last_clean":"2024-04-21T10:29:02.932938+0000","last_became_active":"2024-04-21T10:29:01.947351+0000","last_became_peered":"2024-04-21T10:29:01.947351+0000","last_unstale":"2024-04-21T10:29:02.932938+0000","last_undegraded":"2024-04-21T10:29:02.932938+0000","last_fullsized":"2024-04-21T10:29:02.932938+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:54.538731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.805072+0000","last_change":"2024-04-21T10:29:00.932309+0000","last_active":"2024-04-21T10:29:03.805072+0000","last_peered":"2024-04-21T10:29:03.805072+0000","last_clean":"2024-04-21T10:29:03.805072+0000","last_became_active":"2024-04-21T10:29:00.931935+0000","last_became_peered":"2024-04-21T10:29:00.931935+0000","last_unstale":"2024-04-21T10:29:03.805072+0000","last_undegraded":"2024-04-21T10:29:03.805072+0000","last_fullsized":"2024-04-21T10:29:03.805072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:37:37.604381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097800+0000","last_change":"2024-04-21T10:29:01.946543+0000","last_active":"2024-04-21T10:29:03.097800+0000","last_peered":"2024-04-21T10:29:03.097800+0000","last_clean":"2024-04-21T10:29:03.097800+0000","last_became_active":"2024-04-21T10:29:01.946210+0000","last_became_peered":"2024-04-21T10:29:01.946210+0000","last_unstale":"2024-04-21T10:29:03.097800+0000","last_undegraded":"2024-04-21T10:29:03.097800+0000","last_fullsized":"2024-04-21T10:29:03.097800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:49.368851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:06.817758+0000","last_change":"2024-04-21T10:29:00.924109+0000","last_active":"2024-04-21T10:29:06.817758+0000","last_peered":"2024-04-21T10:29:06.817758+0000","last_clean":"2024-04-21T10:29:06.817758+0000","last_became_active":"2024-04-21T10:29:00.923735+0000","last_became_peered":"2024-04-21T10:29:00.923735+0000","last_unstale":"2024-04-21T10:29:06.817758+0000","last_undegraded":"2024-04-21T10:29:06.817758+0000","last_fullsized":"2024-04-21T10:29:06.817758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:13:54.015774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":23,"num_write":16,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.934404+0000","last_change":"2024-04-21T10:29:01.937211+0000","last_active":"2024-04-21T10:29:02.934404+0000","last_peered":"2024-04-21T10:29:02.934404+0000","last_clean":"2024-04-21T10:29:02.934404+0000","last_became_active":"2024-04-21T10:29:01.936891+0000","last_became_peered":"2024-04-21T10:29:01.936891+0000","last_unstale":"2024-04-21T10:29:02.934404+0000","last_undegraded":"2024-04-21T10:29:02.934404+0000","last_fullsized":"2024-04-21T10:29:02.934404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:26.062236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.816416+0000","last_change":"2024-04-21T10:29:00.920626+0000","last_active":"2024-04-21T10:29:04.816416+0000","last_peered":"2024-04-21T10:29:04.816416+0000","last_clean":"2024-04-21T10:29:04.816416+0000","last_became_active":"2024-04-21T10:29:00.920317+0000","last_became_peered":"2024-04-21T10:29:00.920317+0000","last_unstale":"2024-04-21T10:29:04.816416+0000","last_undegraded":"2024-04-21T10:29:04.816416+0000","last_fullsized":"2024-04-21T10:29:04.816416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:41:18.373403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097461+0000","last_change":"2024-04-21T10:29:01.942494+0000","last_active":"2024-04-21T10:29:03.097461+0000","last_peered":"2024-04-21T10:29:03.097461+0000","last_clean":"2024-04-21T10:29:03.097461+0000","last_became_active":"2024-04-21T10:29:01.942151+0000","last_became_peered":"2024-04-21T10:29:01.942151+0000","last_unstale":"2024-04-21T10:29:03.097461+0000","last_undegraded":"2024-04-21T10:29:03.097461+0000","last_fullsized":"2024-04-21T10:29:03.097461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:39.071158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.807417+0000","last_change":"2024-04-21T10:29:00.928221+0000","last_active":"2024-04-21T10:29:03.807417+0000","last_peered":"2024-04-21T10:29:03.807417+0000","last_clean":"2024-04-21T10:29:03.807417+0000","last_became_active":"2024-04-21T10:29:00.927873+0000","last_became_peered":"2024-04-21T10:29:00.927873+0000","last_unstale":"2024-04-21T10:29:03.807417+0000","last_undegraded":"2024-04-21T10:29:03.807417+0000","last_fullsized":"2024-04-21T10:29:03.807417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:14.898456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.938125+0000","last_change":"2024-04-21T10:29:01.930375+0000","last_active":"2024-04-21T10:29:02.938125+0000","last_peered":"2024-04-21T10:29:02.938125+0000","last_clean":"2024-04-21T10:29:02.938125+0000","last_became_active":"2024-04-21T10:29:01.930009+0000","last_became_peered":"2024-04-21T10:29:01.930009+0000","last_unstale":"2024-04-21T10:29:02.938125+0000","last_undegraded":"2024-04-21T10:29:02.938125+0000","last_fullsized":"2024-04-21T10:29:02.938125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:34.111797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:06.818158+0000","last_change":"2024-04-21T10:29:00.922981+0000","last_active":"2024-04-21T10:29:06.818158+0000","last_peered":"2024-04-21T10:29:06.818158+0000","last_clean":"2024-04-21T10:29:06.818158+0000","last_became_active":"2024-04-21T10:29:00.922668+0000","last_became_peered":"2024-04-21T10:29:00.922668+0000","last_unstale":"2024-04-21T10:29:06.818158+0000","last_undegraded":"2024-04-21T10:29:06.818158+0000","last_fullsized":"2024-04-21T10:29:06.818158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:47.109375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.937635+0000","last_change":"2024-04-21T10:29:01.942530+0000","last_active":"2024-04-21T10:29:02.937635+0000","last_peered":"2024-04-21T10:29:02.937635+0000","last_clean":"2024-04-21T10:29:02.937635+0000","last_became_active":"2024-04-21T10:29:01.942162+0000","last_became_peered":"2024-04-21T10:29:01.942162+0000","last_unstale":"2024-04-21T10:29:02.937635+0000","last_undegraded":"2024-04-21T10:29:02.937635+0000","last_fullsized":"2024-04-21T10:29:02.937635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:46:19.138930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.815197+0000","last_change":"2024-04-21T10:29:00.921104+0000","last_active":"2024-04-21T10:29:04.815197+0000","last_peered":"2024-04-21T10:29:04.815197+0000","last_clean":"2024-04-21T10:29:04.815197+0000","last_became_active":"2024-04-21T10:29:00.920772+0000","last_became_peered":"2024-04-21T10:29:00.920772+0000","last_unstale":"2024-04-21T10:29:04.815197+0000","last_undegraded":"2024-04-21T10:29:04.815197+0000","last_fullsized":"2024-04-21T10:29:04.815197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:23:55.738067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097433+0000","last_change":"2024-04-21T10:29:01.929696+0000","last_active":"2024-04-21T10:29:03.097433+0000","last_peered":"2024-04-21T10:29:03.097433+0000","last_clean":"2024-04-21T10:29:03.097433+0000","last_became_active":"2024-04-21T10:29:01.929328+0000","last_became_peered":"2024-04-21T10:29:01.929328+0000","last_unstale":"2024-04-21T10:29:03.097433+0000","last_undegraded":"2024-04-21T10:29:03.097433+0000","last_fullsized":"2024-04-21T10:29:03.097433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:55:42.735595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:05.808476+0000","last_change":"2024-04-21T10:29:00.925981+0000","last_active":"2024-04-21T10:29:05.808476+0000","last_peered":"2024-04-21T10:29:05.808476+0000","last_clean":"2024-04-21T10:29:05.808476+0000","last_became_active":"2024-04-21T10:29:00.925560+0000","last_became_peered":"2024-04-21T10:29:00.925560+0000","last_unstale":"2024-04-21T10:29:05.808476+0000","last_undegraded":"2024-04-21T10:29:05.808476+0000","last_fullsized":"2024-04-21T10:29:05.808476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:37.467522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097893+0000","last_change":"2024-04-21T10:29:01.941961+0000","last_active":"2024-04-21T10:29:03.097893+0000","last_peered":"2024-04-21T10:29:03.097893+0000","last_clean":"2024-04-21T10:29:03.097893+0000","last_became_active":"2024-04-21T10:29:01.941649+0000","last_became_peered":"2024-04-21T10:29:01.941649+0000","last_unstale":"2024-04-21T10:29:03.097893+0000","last_undegraded":"2024-04-21T10:29:03.097893+0000","last_fullsized":"2024-04-21T10:29:03.097893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:00:56.803856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096656+0000","last_change":"2024-04-21T10:29:00.929504+0000","last_active":"2024-04-21T10:29:03.096656+0000","last_peered":"2024-04-21T10:29:03.096656+0000","last_clean":"2024-04-21T10:29:03.096656+0000","last_became_active":"2024-04-21T10:29:00.929172+0000","last_became_peered":"2024-04-21T10:29:00.929172+0000","last_unstale":"2024-04-21T10:29:03.096656+0000","last_undegraded":"2024-04-21T10:29:03.096656+0000","last_fullsized":"2024-04-21T10:29:03.096656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:40.512175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098134+0000","last_change":"2024-04-21T10:29:01.939333+0000","last_active":"2024-04-21T10:29:03.098134+0000","last_peered":"2024-04-21T10:29:03.098134+0000","last_clean":"2024-04-21T10:29:03.098134+0000","last_became_active":"2024-04-21T10:29:01.938924+0000","last_became_peered":"2024-04-21T10:29:01.938924+0000","last_unstale":"2024-04-21T10:29:03.098134+0000","last_undegraded":"2024-04-21T10:29:03.098134+0000","last_fullsized":"2024-04-21T10:29:03.098134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:20:01.287604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.937646+0000","last_change":"2024-04-21T10:29:00.927455+0000","last_active":"2024-04-21T10:29:02.937646+0000","last_peered":"2024-04-21T10:29:02.937646+0000","last_clean":"2024-04-21T10:29:02.937646+0000","last_became_active":"2024-04-21T10:29:00.927180+0000","last_became_peered":"2024-04-21T10:29:00.927180+0000","last_unstale":"2024-04-21T10:29:02.937646+0000","last_undegraded":"2024-04-21T10:29:02.937646+0000","last_fullsized":"2024-04-21T10:29:02.937646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:11:57.166607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097261+0000","last_change":"2024-04-21T10:29:01.937646+0000","last_active":"2024-04-21T10:29:03.097261+0000","last_peered":"2024-04-21T10:29:03.097261+0000","last_clean":"2024-04-21T10:29:03.097261+0000","last_became_active":"2024-04-21T10:29:01.937216+0000","last_became_peered":"2024-04-21T10:29:01.937216+0000","last_unstale":"2024-04-21T10:29:03.097261+0000","last_undegraded":"2024-04-21T10:29:03.097261+0000","last_fullsized":"2024-04-21T10:29:03.097261+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:38.265252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097258+0000","last_change":"2024-04-21T10:29:01.926860+0000","last_active":"2024-04-21T10:29:03.097258+0000","last_peered":"2024-04-21T10:29:03.097258+0000","last_clean":"2024-04-21T10:29:03.097258+0000","last_became_active":"2024-04-21T10:29:01.925951+0000","last_became_peered":"2024-04-21T10:29:01.925951+0000","last_unstale":"2024-04-21T10:29:03.097258+0000","last_undegraded":"2024-04-21T10:29:03.097258+0000","last_fullsized":"2024-04-21T10:29:03.097258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:01:55.127301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.814347+0000","last_change":"2024-04-21T10:29:00.926502+0000","last_active":"2024-04-21T10:29:04.814347+0000","last_peered":"2024-04-21T10:29:04.814347+0000","last_clean":"2024-04-21T10:29:04.814347+0000","last_became_active":"2024-04-21T10:29:00.926175+0000","last_became_peered":"2024-04-21T10:29:00.926175+0000","last_unstale":"2024-04-21T10:29:04.814347+0000","last_undegraded":"2024-04-21T10:29:04.814347+0000","last_fullsized":"2024-04-21T10:29:04.814347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:40:34.774085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097956+0000","last_change":"2024-04-21T10:29:01.926369+0000","last_active":"2024-04-21T10:29:03.097956+0000","last_peered":"2024-04-21T10:29:03.097956+0000","last_clean":"2024-04-21T10:29:03.097956+0000","last_became_active":"2024-04-21T10:29:01.926011+0000","last_became_peered":"2024-04-21T10:29:01.926011+0000","last_unstale":"2024-04-21T10:29:03.097956+0000","last_undegraded":"2024-04-21T10:29:03.097956+0000","last_fullsized":"2024-04-21T10:29:03.097956+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:43:18.273057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:05.812932+0000","last_change":"2024-04-21T10:29:00.921748+0000","last_active":"2024-04-21T10:29:05.812932+0000","last_peered":"2024-04-21T10:29:05.812932+0000","last_clean":"2024-04-21T10:29:05.812932+0000","last_became_active":"2024-04-21T10:29:00.921408+0000","last_became_peered":"2024-04-21T10:29:00.921408+0000","last_unstale":"2024-04-21T10:29:05.812932+0000","last_undegraded":"2024-04-21T10:29:05.812932+0000","last_fullsized":"2024-04-21T10:29:05.812932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:52:27.606280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":351,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.937503+0000","last_change":"2024-04-21T10:27:49.787009+0000","last_active":"2024-04-21T10:29:02.937503+0000","last_peered":"2024-04-21T10:29:02.937503+0000","last_clean":"2024-04-21T10:29:02.937503+0000","last_became_active":"2024-04-21T10:27:49.786530+0000","last_became_peered":"2024-04-21T10:27:49.786530+0000","last_unstale":"2024-04-21T10:29:02.937503+0000","last_undegraded":"2024-04-21T10:29:02.937503+0000","last_fullsized":"2024-04-21T10:29:02.937503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_clean_scrub_stamp":"2024-04-21T10:24:28.367927+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:20:12.106231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096569+0000","last_change":"2024-04-21T10:29:01.929293+0000","last_active":"2024-04-21T10:29:03.096569+0000","last_peered":"2024-04-21T10:29:03.096569+0000","last_clean":"2024-04-21T10:29:03.096569+0000","last_became_active":"2024-04-21T10:29:01.928942+0000","last_became_peered":"2024-04-21T10:29:01.928942+0000","last_unstale":"2024-04-21T10:29:03.096569+0000","last_undegraded":"2024-04-21T10:29:03.096569+0000","last_fullsized":"2024-04-21T10:29:03.096569+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:35.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.813224+0000","last_change":"2024-04-21T10:29:00.927521+0000","last_active":"2024-04-21T10:29:04.813224+0000","last_peered":"2024-04-21T10:29:04.813224+0000","last_clean":"2024-04-21T10:29:04.813224+0000","last_became_active":"2024-04-21T10:29:00.927202+0000","last_became_peered":"2024-04-21T10:29:00.927202+0000","last_unstale":"2024-04-21T10:29:04.813224+0000","last_undegraded":"2024-04-21T10:29:04.813224+0000","last_fullsized":"2024-04-21T10:29:04.813224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:11:32.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.804026+0000","last_change":"2024-04-21T10:29:00.930728+0000","last_active":"2024-04-21T10:29:03.804026+0000","last_peered":"2024-04-21T10:29:03.804026+0000","last_clean":"2024-04-21T10:29:03.804026+0000","last_became_active":"2024-04-21T10:29:00.930457+0000","last_became_peered":"2024-04-21T10:29:00.930457+0000","last_unstale":"2024-04-21T10:29:03.804026+0000","last_undegraded":"2024-04-21T10:29:03.804026+0000","last_fullsized":"2024-04-21T10:29:03.804026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:28.032954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097702+0000","last_change":"2024-04-21T10:29:01.932278+0000","last_active":"2024-04-21T10:29:03.097702+0000","last_peered":"2024-04-21T10:29:03.097702+0000","last_clean":"2024-04-21T10:29:03.097702+0000","last_became_active":"2024-04-21T10:29:01.931774+0000","last_became_peered":"2024-04-21T10:29:01.931774+0000","last_unstale":"2024-04-21T10:29:03.097702+0000","last_undegraded":"2024-04-21T10:29:03.097702+0000","last_fullsized":"2024-04-21T10:29:03.097702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:34:25.858475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.813481+0000","last_change":"2024-04-21T10:29:00.926071+0000","last_active":"2024-04-21T10:29:04.813481+0000","last_peered":"2024-04-21T10:29:04.813481+0000","last_clean":"2024-04-21T10:29:04.813481+0000","last_became_active":"2024-04-21T10:29:00.925607+0000","last_became_peered":"2024-04-21T10:29:00.925607+0000","last_unstale":"2024-04-21T10:29:04.813481+0000","last_undegraded":"2024-04-21T10:29:04.813481+0000","last_fullsized":"2024-04-21T10:29:04.813481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:26.350652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096783+0000","last_change":"2024-04-21T10:29:01.941365+0000","last_active":"2024-04-21T10:29:03.096783+0000","last_peered":"2024-04-21T10:29:03.096783+0000","last_clean":"2024-04-21T10:29:03.096783+0000","last_became_active":"2024-04-21T10:29:01.940102+0000","last_became_peered":"2024-04-21T10:29:01.940102+0000","last_unstale":"2024-04-21T10:29:03.096783+0000","last_undegraded":"2024-04-21T10:29:03.096783+0000","last_fullsized":"2024-04-21T10:29:03.096783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:21.976855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.803180+0000","last_change":"2024-04-21T10:29:00.924322+0000","last_active":"2024-04-21T10:29:03.803180+0000","last_peered":"2024-04-21T10:29:03.803180+0000","last_clean":"2024-04-21T10:29:03.803180+0000","last_became_active":"2024-04-21T10:29:00.924016+0000","last_became_peered":"2024-04-21T10:29:00.924016+0000","last_unstale":"2024-04-21T10:29:03.803180+0000","last_undegraded":"2024-04-21T10:29:03.803180+0000","last_fullsized":"2024-04-21T10:29:03.803180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:49:24.004951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932773+0000","last_change":"2024-04-21T10:29:01.946315+0000","last_active":"2024-04-21T10:29:02.932773+0000","last_peered":"2024-04-21T10:29:02.932773+0000","last_clean":"2024-04-21T10:29:02.932773+0000","last_became_active":"2024-04-21T10:29:01.945920+0000","last_became_peered":"2024-04-21T10:29:01.945920+0000","last_unstale":"2024-04-21T10:29:02.932773+0000","last_undegraded":"2024-04-21T10:29:02.932773+0000","last_fullsized":"2024-04-21T10:29:02.932773+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:10:52.361615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097006+0000","last_change":"2024-04-21T10:29:00.919086+0000","last_active":"2024-04-21T10:29:03.097006+0000","last_peered":"2024-04-21T10:29:03.097006+0000","last_clean":"2024-04-21T10:29:03.097006+0000","last_became_active":"2024-04-21T10:29:00.918727+0000","last_became_peered":"2024-04-21T10:29:00.918727+0000","last_unstale":"2024-04-21T10:29:03.097006+0000","last_undegraded":"2024-04-21T10:29:03.097006+0000","last_fullsized":"2024-04-21T10:29:03.097006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:53:27.209135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.934128+0000","last_change":"2024-04-21T10:29:01.944037+0000","last_active":"2024-04-21T10:29:02.934128+0000","last_peered":"2024-04-21T10:29:02.934128+0000","last_clean":"2024-04-21T10:29:02.934128+0000","last_became_active":"2024-04-21T10:29:01.943692+0000","last_became_peered":"2024-04-21T10:29:01.943692+0000","last_unstale":"2024-04-21T10:29:02.934128+0000","last_undegraded":"2024-04-21T10:29:02.934128+0000","last_fullsized":"2024-04-21T10:29:02.934128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:51.285535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.804286+0000","last_change":"2024-04-21T10:29:00.930937+0000","last_active":"2024-04-21T10:29:03.804286+0000","last_peered":"2024-04-21T10:29:03.804286+0000","last_clean":"2024-04-21T10:29:03.804286+0000","last_became_active":"2024-04-21T10:29:00.930348+0000","last_became_peered":"2024-04-21T10:29:00.930348+0000","last_unstale":"2024-04-21T10:29:03.804286+0000","last_undegraded":"2024-04-21T10:29:03.804286+0000","last_fullsized":"2024-04-21T10:29:03.804286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:43:33.801432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096986+0000","last_change":"2024-04-21T10:29:01.937116+0000","last_active":"2024-04-21T10:29:03.096986+0000","last_peered":"2024-04-21T10:29:03.096986+0000","last_clean":"2024-04-21T10:29:03.096986+0000","last_became_active":"2024-04-21T10:29:01.936756+0000","last_became_peered":"2024-04-21T10:29:01.936756+0000","last_unstale":"2024-04-21T10:29:03.096986+0000","last_undegraded":"2024-04-21T10:29:03.096986+0000","last_fullsized":"2024-04-21T10:29:03.096986+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:02.732399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098255+0000","last_change":"2024-04-21T10:29:00.919355+0000","last_active":"2024-04-21T10:29:03.098255+0000","last_peered":"2024-04-21T10:29:03.098255+0000","last_clean":"2024-04-21T10:29:03.098255+0000","last_became_active":"2024-04-21T10:29:00.919095+0000","last_became_peered":"2024-04-21T10:29:00.919095+0000","last_unstale":"2024-04-21T10:29:03.098255+0000","last_undegraded":"2024-04-21T10:29:03.098255+0000","last_fullsized":"2024-04-21T10:29:03.098255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:07:34.821171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097755+0000","last_change":"2024-04-21T10:29:01.940677+0000","last_active":"2024-04-21T10:29:03.097755+0000","last_peered":"2024-04-21T10:29:03.097755+0000","last_clean":"2024-04-21T10:29:03.097755+0000","last_became_active":"2024-04-21T10:29:01.940296+0000","last_became_peered":"2024-04-21T10:29:01.940296+0000","last_unstale":"2024-04-21T10:29:03.097755+0000","last_undegraded":"2024-04-21T10:29:03.097755+0000","last_fullsized":"2024-04-21T10:29:03.097755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:24:52.662041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.806404+0000","last_change":"2024-04-21T10:29:00.924392+0000","last_active":"2024-04-21T10:29:03.806404+0000","last_peered":"2024-04-21T10:29:03.806404+0000","last_clean":"2024-04-21T10:29:03.806404+0000","last_became_active":"2024-04-21T10:29:00.924029+0000","last_became_peered":"2024-04-21T10:29:00.924029+0000","last_unstale":"2024-04-21T10:29:03.806404+0000","last_undegraded":"2024-04-21T10:29:03.806404+0000","last_fullsized":"2024-04-21T10:29:03.806404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:23:50.726978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.937954+0000","last_change":"2024-04-21T10:29:01.939687+0000","last_active":"2024-04-21T10:29:02.937954+0000","last_peered":"2024-04-21T10:29:02.937954+0000","last_clean":"2024-04-21T10:29:02.937954+0000","last_became_active":"2024-04-21T10:29:01.939343+0000","last_became_peered":"2024-04-21T10:29:01.939343+0000","last_unstale":"2024-04-21T10:29:02.937954+0000","last_undegraded":"2024-04-21T10:29:02.937954+0000","last_fullsized":"2024-04-21T10:29:02.937954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:50.481998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:04.816898+0000","last_change":"2024-04-21T10:29:00.923115+0000","last_active":"2024-04-21T10:29:04.816898+0000","last_peered":"2024-04-21T10:29:04.816898+0000","last_clean":"2024-04-21T10:29:04.816898+0000","last_became_active":"2024-04-21T10:29:00.922707+0000","last_became_peered":"2024-04-21T10:29:00.922707+0000","last_unstale":"2024-04-21T10:29:04.816898+0000","last_undegraded":"2024-04-21T10:29:04.816898+0000","last_fullsized":"2024-04-21T10:29:04.816898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:55:53.419904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.938105+0000","last_change":"2024-04-21T10:29:01.940666+0000","last_active":"2024-04-21T10:29:02.938105+0000","last_peered":"2024-04-21T10:29:02.938105+0000","last_clean":"2024-04-21T10:29:02.938105+0000","last_became_active":"2024-04-21T10:29:01.940331+0000","last_became_peered":"2024-04-21T10:29:01.940331+0000","last_unstale":"2024-04-21T10:29:02.938105+0000","last_undegraded":"2024-04-21T10:29:02.938105+0000","last_fullsized":"2024-04-21T10:29:02.938105+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:52:41.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098230+0000","last_change":"2024-04-21T10:29:00.927873+0000","last_active":"2024-04-21T10:29:03.098230+0000","last_peered":"2024-04-21T10:29:03.098230+0000","last_clean":"2024-04-21T10:29:03.098230+0000","last_became_active":"2024-04-21T10:29:00.927403+0000","last_became_peered":"2024-04-21T10:29:00.927403+0000","last_unstale":"2024-04-21T10:29:03.098230+0000","last_undegraded":"2024-04-21T10:29:03.098230+0000","last_fullsized":"2024-04-21T10:29:03.098230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:05:05.276978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.937854+0000","last_change":"2024-04-21T10:29:01.944725+0000","last_active":"2024-04-21T10:29:02.937854+0000","last_peered":"2024-04-21T10:29:02.937854+0000","last_clean":"2024-04-21T10:29:02.937854+0000","last_became_active":"2024-04-21T10:29:01.944326+0000","last_became_peered":"2024-04-21T10:29:01.944326+0000","last_unstale":"2024-04-21T10:29:02.937854+0000","last_undegraded":"2024-04-21T10:29:02.937854+0000","last_fullsized":"2024-04-21T10:29:02.937854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:50.209064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096952+0000","last_change":"2024-04-21T10:29:00.924694+0000","last_active":"2024-04-21T10:29:03.096952+0000","last_peered":"2024-04-21T10:29:03.096952+0000","last_clean":"2024-04-21T10:29:03.096952+0000","last_became_active":"2024-04-21T10:29:00.924373+0000","last_became_peered":"2024-04-21T10:29:00.924373+0000","last_unstale":"2024-04-21T10:29:03.096952+0000","last_undegraded":"2024-04-21T10:29:03.096952+0000","last_fullsized":"2024-04-21T10:29:03.096952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:10:14.681630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.938194+0000","last_change":"2024-04-21T10:29:01.939277+0000","last_active":"2024-04-21T10:29:02.938194+0000","last_peered":"2024-04-21T10:29:02.938194+0000","last_clean":"2024-04-21T10:29:02.938194+0000","last_became_active":"2024-04-21T10:29:01.938844+0000","last_became_peered":"2024-04-21T10:29:01.938844+0000","last_unstale":"2024-04-21T10:29:02.938194+0000","last_undegraded":"2024-04-21T10:29:02.938194+0000","last_fullsized":"2024-04-21T10:29:02.938194+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:38:14.553868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097181+0000","last_change":"2024-04-21T10:29:01.942654+0000","last_active":"2024-04-21T10:29:03.097181+0000","last_peered":"2024-04-21T10:29:03.097181+0000","last_clean":"2024-04-21T10:29:03.097181+0000","last_became_active":"2024-04-21T10:29:01.942324+0000","last_became_peered":"2024-04-21T10:29:01.942324+0000","last_unstale":"2024-04-21T10:29:03.097181+0000","last_undegraded":"2024-04-21T10:29:03.097181+0000","last_fullsized":"2024-04-21T10:29:03.097181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:19.027138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098173+0000","last_change":"2024-04-21T10:29:01.943088+0000","last_active":"2024-04-21T10:29:03.098173+0000","last_peered":"2024-04-21T10:29:03.098173+0000","last_clean":"2024-04-21T10:29:03.098173+0000","last_became_active":"2024-04-21T10:29:01.942542+0000","last_became_peered":"2024-04-21T10:29:01.942542+0000","last_unstale":"2024-04-21T10:29:03.098173+0000","last_undegraded":"2024-04-21T10:29:03.098173+0000","last_fullsized":"2024-04-21T10:29:03.098173+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:50:55.283349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098011+0000","last_change":"2024-04-21T10:29:01.944405+0000","last_active":"2024-04-21T10:29:03.098011+0000","last_peered":"2024-04-21T10:29:03.098011+0000","last_clean":"2024-04-21T10:29:03.098011+0000","last_became_active":"2024-04-21T10:29:01.944085+0000","last_became_peered":"2024-04-21T10:29:01.944085+0000","last_unstale":"2024-04-21T10:29:03.098011+0000","last_undegraded":"2024-04-21T10:29:03.098011+0000","last_fullsized":"2024-04-21T10:29:03.098011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:13.299859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:02.932624+0000","last_change":"2024-04-21T10:29:01.941054+0000","last_active":"2024-04-21T10:29:02.932624+0000","last_peered":"2024-04-21T10:29:02.932624+0000","last_clean":"2024-04-21T10:29:02.932624+0000","last_became_active":"2024-04-21T10:29:01.940243+0000","last_became_peered":"2024-04-21T10:29:01.940243+0000","last_unstale":"2024-04-21T10:29:02.932624+0000","last_undegraded":"2024-04-21T10:29:02.932624+0000","last_fullsized":"2024-04-21T10:29:02.932624+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:31:04.222026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097638+0000","last_change":"2024-04-21T10:29:01.934315+0000","last_active":"2024-04-21T10:29:03.097638+0000","last_peered":"2024-04-21T10:29:03.097638+0000","last_clean":"2024-04-21T10:29:03.097638+0000","last_became_active":"2024-04-21T10:29:01.933996+0000","last_became_peered":"2024-04-21T10:29:01.933996+0000","last_unstale":"2024-04-21T10:29:03.097638+0000","last_undegraded":"2024-04-21T10:29:03.097638+0000","last_fullsized":"2024-04-21T10:29:03.097638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:08:07.374591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097903+0000","last_change":"2024-04-21T10:29:01.947268+0000","last_active":"2024-04-21T10:29:03.097903+0000","last_peered":"2024-04-21T10:29:03.097903+0000","last_clean":"2024-04-21T10:29:03.097903+0000","last_became_active":"2024-04-21T10:29:01.946921+0000","last_became_peered":"2024-04-21T10:29:01.946921+0000","last_unstale":"2024-04-21T10:29:03.097903+0000","last_undegraded":"2024-04-21T10:29:03.097903+0000","last_fullsized":"2024-04-21T10:29:03.097903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:50:43.784920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.098783+0000","last_change":"2024-04-21T10:29:01.949969+0000","last_active":"2024-04-21T10:29:03.098783+0000","last_peered":"2024-04-21T10:29:03.098783+0000","last_clean":"2024-04-21T10:29:03.098783+0000","last_became_active":"2024-04-21T10:29:01.949487+0000","last_became_peered":"2024-04-21T10:29:01.949487+0000","last_unstale":"2024-04-21T10:29:03.098783+0000","last_undegraded":"2024-04-21T10:29:03.098783+0000","last_fullsized":"2024-04-21T10:29:03.098783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:26:59.620209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.096650+0000","last_change":"2024-04-21T10:29:01.937904+0000","last_active":"2024-04-21T10:29:03.096650+0000","last_peered":"2024-04-21T10:29:03.096650+0000","last_clean":"2024-04-21T10:29:03.096650+0000","last_became_active":"2024-04-21T10:29:01.937504+0000","last_became_peered":"2024-04-21T10:29:01.937504+0000","last_unstale":"2024-04-21T10:29:03.096650+0000","last_undegraded":"2024-04-21T10:29:03.096650+0000","last_fullsized":"2024-04-21T10:29:03.096650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:58:24.245118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097183+0000","last_change":"2024-04-21T10:29:01.934320+0000","last_active":"2024-04-21T10:29:03.097183+0000","last_peered":"2024-04-21T10:29:03.097183+0000","last_clean":"2024-04-21T10:29:03.097183+0000","last_became_active":"2024-04-21T10:29:01.934015+0000","last_became_peered":"2024-04-21T10:29:01.934015+0000","last_unstale":"2024-04-21T10:29:03.097183+0000","last_undegraded":"2024-04-21T10:29:03.097183+0000","last_fullsized":"2024-04-21T10:29:03.097183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:31:39.700427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097341+0000","last_change":"2024-04-21T10:29:01.936880+0000","last_active":"2024-04-21T10:29:03.097341+0000","last_peered":"2024-04-21T10:29:03.097341+0000","last_clean":"2024-04-21T10:29:03.097341+0000","last_became_active":"2024-04-21T10:29:01.936555+0000","last_became_peered":"2024-04-21T10:29:01.936555+0000","last_unstale":"2024-04-21T10:29:03.097341+0000","last_undegraded":"2024-04-21T10:29:03.097341+0000","last_fullsized":"2024-04-21T10:29:03.097341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:32.831606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097772+0000","last_change":"2024-04-21T10:29:01.943238+0000","last_active":"2024-04-21T10:29:03.097772+0000","last_peered":"2024-04-21T10:29:03.097772+0000","last_clean":"2024-04-21T10:29:03.097772+0000","last_became_active":"2024-04-21T10:29:01.942841+0000","last_became_peered":"2024-04-21T10:29:01.942841+0000","last_unstale":"2024-04-21T10:29:03.097772+0000","last_undegraded":"2024-04-21T10:29:03.097772+0000","last_fullsized":"2024-04-21T10:29:03.097772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:59:48.551524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097387+0000","last_change":"2024-04-21T10:29:01.933712+0000","last_active":"2024-04-21T10:29:03.097387+0000","last_peered":"2024-04-21T10:29:03.097387+0000","last_clean":"2024-04-21T10:29:03.097387+0000","last_became_active":"2024-04-21T10:29:01.933399+0000","last_became_peered":"2024-04-21T10:29:01.933399+0000","last_unstale":"2024-04-21T10:29:03.097387+0000","last_undegraded":"2024-04-21T10:29:03.097387+0000","last_fullsized":"2024-04-21T10:29:03.097387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:34:16.346403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-21T10:29:03.097336+0000","last_change":"2024-04-21T10:29:01.947225+0000","last_active":"2024-04-21T10:29:03.097336+0000","last_peered":"2024-04-21T10:29:03.097336+0000","last_clean":"2024-04-21T10:29:03.097336+0000","last_became_active":"2024-04-21T10:29:01.946890+0000","last_became_peered":"2024-04-21T10:29:01.946890+0000","last_unstale":"2024-04-21T10:29:03.097336+0000","last_undegraded":"2024-04-21T10:29:03.097336+0000","last_fullsized":"2024-04-21T10:29:03.097336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:19:00.960767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24896,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":33,"num_read_kb":45,"num_write":130,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":72312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":446648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1046615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743448,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":446884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005085,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":446804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266721,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1037638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":446156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":439108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":447544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":468311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":446350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":447101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691885,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32696,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712456,"statfs":{"total":95995035648,"available":95961554944,"internally_reserved":0,"allocated":1556480,"data_stored":1037910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":39543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T10:29:09.028 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-21T10:29:09.029 INFO:tasks.cephadm:Setting up client nodes... 2024-04-21T10:29:09.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-21T10:29:09.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:08 smithi190 ceph-mon[31423]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:09.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:08 smithi190 ceph-mon[31423]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:09.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:08 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3284055230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:29:09.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:08 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3608386613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:09.309 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:09.427 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:08 smithi026 ceph-mon[27965]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:09.427 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:08 smithi026 ceph-mon[27965]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:09.427 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:08 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3284055230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:29:09.427 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:08 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3608386613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:09.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:08 smithi169 ceph-mon[31469]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:09.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:08 smithi169 ceph-mon[31469]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:29:09.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:08 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3284055230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:29:09.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:08 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3608386613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:10.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:09 smithi026 ceph-mon[27965]: pgmap v290: 97 pgs: 97 active+clean; 601 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.4 KiB/s wr, 22 op/s 2024-04-21T10:29:10.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:09 smithi026 ceph-mon[27965]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:29:10.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:09 smithi190 ceph-mon[31423]: pgmap v290: 97 pgs: 97 active+clean; 601 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.4 KiB/s wr, 22 op/s 2024-04-21T10:29:10.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:09 smithi190 ceph-mon[31423]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:29:10.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:09 smithi169 ceph-mon[31469]: pgmap v290: 97 pgs: 97 active+clean; 601 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.4 KiB/s wr, 22 op/s 2024-04-21T10:29:10.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:09 smithi169 ceph-mon[31469]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:29:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.941+0000 7f47b739f700 1 -- 172.21.15.26:0/4043847851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b0102af0 msgr2=0x7f47b0102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.941+0000 7f47b739f700 1 --2- 172.21.15.26:0/4043847851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b0102af0 0x7f47b0102f10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0099c0 tx=0x7f47ac009cd0 comp rx=0 tx=0).stop 2024-04-21T10:29:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.941+0000 7f47b739f700 1 -- 172.21.15.26:0/4043847851 shutdown_connections 2024-04-21T10:29:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.941+0000 7f47b739f700 1 --2- 172.21.15.26:0/4043847851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47b0104690 0x7f47b010af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.941+0000 7f47b739f700 1 --2- 172.21.15.26:0/4043847851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b0102af0 0x7f47b0102f10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.941+0000 7f47b739f700 1 --2- 172.21.15.26:0/4043847851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47b0103cf0 0x7f47b0104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.941+0000 7f47b739f700 1 -- 172.21.15.26:0/4043847851 >> 172.21.15.26:0/4043847851 conn(0x7f47b00fe090 msgr2=0x7f47b01004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.942+0000 7f47b739f700 1 -- 172.21.15.26:0/4043847851 shutdown_connections 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.942+0000 7f47b739f700 1 -- 172.21.15.26:0/4043847851 wait complete. 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.942+0000 7f47b739f700 1 Processor -- start 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.942+0000 7f47b739f700 1 -- start start 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.942+0000 7f47b739f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47b0102af0 0x7f47b0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.942+0000 7f47b739f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47b0103cf0 0x7f47b01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.942+0000 7f47b739f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b0104690 0x7f47b01a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.942+0000 7f47b739f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47b010e600 con 0x7f47b0104690 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.942+0000 7f47b739f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f47b010e480 con 0x7f47b0102af0 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.942+0000 7f47b739f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f47b010e780 con 0x7f47b0103cf0 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b593c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b0104690 0x7f47b01a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b593c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b0104690 0x7f47b01a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58970/0 (socket says 172.21.15.26:58970) 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b593c700 1 -- 172.21.15.26:0/157433022 learned_addr learned my addr 172.21.15.26:0/157433022 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b513b700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47b0102af0 0x7f47b0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b493a700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47b0103cf0 0x7f47b01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:10.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b593c700 1 -- 172.21.15.26:0/157433022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47b0103cf0 msgr2=0x7f47b01990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:10.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b593c700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47b0103cf0 0x7f47b01990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:10.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b593c700 1 -- 172.21.15.26:0/157433022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47b0102af0 msgr2=0x7f47b0198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:10.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b593c700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47b0102af0 0x7f47b0198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:10.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b593c700 1 -- 172.21.15.26:0/157433022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b01a0810 con 0x7f47b0104690 2024-04-21T10:29:10.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b593c700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b0104690 0x7f47b01a0170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f47a8007f10 tx=0x7f47a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:10.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47a67fc700 1 -- 172.21.15.26:0/157433022 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8004bb0 con 0x7f47b0104690 2024-04-21T10:29:10.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47a67fc700 1 -- 172.21.15.26:0/157433022 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a8004d50 con 0x7f47b0104690 2024-04-21T10:29:10.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b739f700 1 -- 172.21.15.26:0/157433022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b01a0b00 con 0x7f47b0104690 2024-04-21T10:29:10.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47a67fc700 1 -- 172.21.15.26:0/157433022 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a80058e0 con 0x7f47b0104690 2024-04-21T10:29:10.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.943+0000 7f47b739f700 1 -- 172.21.15.26:0/157433022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b01a1040 con 0x7f47b0104690 2024-04-21T10:29:10.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.944+0000 7f47b739f700 1 -- 172.21.15.26:0/157433022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b0050f80 con 0x7f47b0104690 2024-04-21T10:29:10.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.946+0000 7f47a67fc700 1 -- 172.21.15.26:0/157433022 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f47a801a020 con 0x7f47b0104690 2024-04-21T10:29:10.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.946+0000 7f47a67fc700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f479c0a3d80 0x7f479c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:10.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.946+0000 7f47b513b700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f479c0a3d80 0x7f479c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:10.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.947+0000 7f47a67fc700 1 -- 172.21.15.26:0/157433022 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f47a80cfa90 con 0x7f47b0104690 2024-04-21T10:29:10.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.947+0000 7f47b513b700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f479c0a3d80 0x7f479c0a6240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0038c0 tx=0x7f47ac00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:10.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:10.951+0000 7f47a67fc700 1 -- 172.21.15.26:0/157433022 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47a8010070 con 0x7f47b0104690 2024-04-21T10:29:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.246+0000 7f47b739f700 1 -- 172.21.15.26:0/157433022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f47b002e000 con 0x7f47b0104690 2024-04-21T10:29:11.246 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:10 smithi026 ceph-mon[27965]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:29:11.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.248+0000 7f47a67fc700 1 -- 172.21.15.26:0/157433022 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f47a809c4c0 con 0x7f47b0104690 2024-04-21T10:29:11.248 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-04-21T10:29:11.248 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQB36iRmrvvCDhAAJUjkRosBPBWcR0EfYoznQQ== 2024-04-21T10:29:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.251+0000 7f47b739f700 1 -- 172.21.15.26:0/157433022 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f479c0a3d80 msgr2=0x7f479c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.251+0000 7f47b739f700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f479c0a3d80 0x7f479c0a6240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0038c0 tx=0x7f47ac00b540 comp rx=0 tx=0).stop 2024-04-21T10:29:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.251+0000 7f47b739f700 1 -- 172.21.15.26:0/157433022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b0104690 msgr2=0x7f47b01a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.251+0000 7f47b739f700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b0104690 0x7f47b01a0170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f47a8007f10 tx=0x7f47a800d3b0 comp rx=0 tx=0).stop 2024-04-21T10:29:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.251+0000 7f47b739f700 1 -- 172.21.15.26:0/157433022 shutdown_connections 2024-04-21T10:29:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.251+0000 7f47b739f700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f479c0a3d80 0x7f479c0a6240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.251+0000 7f47b739f700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47b0102af0 0x7f47b0198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.251+0000 7f47b739f700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b0104690 0x7f47b01a0170 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.251+0000 7f47b739f700 1 --2- 172.21.15.26:0/157433022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47b0103cf0 0x7f47b01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.251+0000 7f47b739f700 1 -- 172.21.15.26:0/157433022 >> 172.21.15.26:0/157433022 conn(0x7f47b00fe090 msgr2=0x7f47b0106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.251+0000 7f47b739f700 1 -- 172.21.15.26:0/157433022 shutdown_connections 2024-04-21T10:29:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:11.252+0000 7f47b739f700 1 -- 172.21.15.26:0/157433022 wait complete. 2024-04-21T10:29:11.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:10 smithi190 ceph-mon[31423]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:29:11.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:10 smithi169 ceph-mon[31469]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:29:11.855 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:29:11.855 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-21T10:29:11.855 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-21T10:29:11.892 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-21T10:29:11.892 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-21T10:29:11.892 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-21T10:29:12.179 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:12.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:11 smithi026 ceph-mon[27965]: pgmap v291: 97 pgs: 97 active+clean; 601 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.4 KiB/s wr, 20 op/s 2024-04-21T10:29:12.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:11 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/157433022' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T10:29:12.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:11 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/157433022' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T10:29:12.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:11 smithi190 ceph-mon[31423]: pgmap v291: 97 pgs: 97 active+clean; 601 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.4 KiB/s wr, 20 op/s 2024-04-21T10:29:12.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:11 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/157433022' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T10:29:12.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:11 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/157433022' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T10:29:12.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:11 smithi169 ceph-mon[31469]: pgmap v291: 97 pgs: 97 active+clean; 601 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.4 KiB/s wr, 20 op/s 2024-04-21T10:29:12.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:11 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/157433022' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T10:29:12.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:11 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/157433022' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T10:29:13.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.723+0000 7f04aff88700 1 -- 172.21.15.26:0/3398243069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a8103450 msgr2=0x7f04a810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.724+0000 7f04aff88700 1 --2- 172.21.15.26:0/3398243069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a8103450 0x7f04a810f8b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04a000b0a0 tx=0x7f04a000b3b0 comp rx=0 tx=0).stop 2024-04-21T10:29:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.724+0000 7f04aff88700 1 -- 172.21.15.26:0/3398243069 shutdown_connections 2024-04-21T10:29:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.724+0000 7f04aff88700 1 --2- 172.21.15.26:0/3398243069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a8103450 0x7f04a810f8b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.724+0000 7f04aff88700 1 --2- 172.21.15.26:0/3398243069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a8108a50 0x7f04a8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.724+0000 7f04aff88700 1 --2- 172.21.15.26:0/3398243069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a8102a90 0x7f04a8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.724+0000 7f04aff88700 1 -- 172.21.15.26:0/3398243069 >> 172.21.15.26:0/3398243069 conn(0x7f04a8079ba0 msgr2=0x7f04a8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.724+0000 7f04aff88700 1 -- 172.21.15.26:0/3398243069 shutdown_connections 2024-04-21T10:29:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.724+0000 7f04aff88700 1 -- 172.21.15.26:0/3398243069 wait complete. 2024-04-21T10:29:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.724+0000 7f04aff88700 1 Processor -- start 2024-04-21T10:29:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.724+0000 7f04aff88700 1 -- start start 2024-04-21T10:29:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04aff88700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a8102a90 0x7f04a8199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04aff88700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a8103450 0x7f04a8199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04aff88700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a8108a50 0x7f04a819d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04aff88700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04a8112c00 con 0x7f04a8108a50 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04aff88700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04a8112a80 con 0x7f04a8103450 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04aff88700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f04a8112d80 con 0x7f04a8102a90 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04ae525700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a8108a50 0x7f04a819d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04ae525700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a8108a50 0x7f04a819d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58974/0 (socket says 172.21.15.26:58974) 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04ae525700 1 -- 172.21.15.26:0/704116605 learned_addr learned my addr 172.21.15.26:0/704116605 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04add24700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a8102a90 0x7f04a8199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04ad523700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a8103450 0x7f04a8199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04ae525700 1 -- 172.21.15.26:0/704116605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a8102a90 msgr2=0x7f04a8199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04ae525700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a8102a90 0x7f04a8199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04ae525700 1 -- 172.21.15.26:0/704116605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a8103450 msgr2=0x7f04a8199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:13.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04ae525700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a8103450 0x7f04a8199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04ae525700 1 -- 172.21.15.26:0/704116605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a819dfd0 con 0x7f04a8108a50 2024-04-21T10:29:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04add24700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a8102a90 0x7f04a8199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:29:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.725+0000 7f04ae525700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a8108a50 0x7f04a819d890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f04a00062a0 tx=0x7f04a0004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.726+0000 7f049effd700 1 -- 172.21.15.26:0/704116605 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0021070 con 0x7f04a8108a50 2024-04-21T10:29:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.726+0000 7f04aff88700 1 -- 172.21.15.26:0/704116605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a819e260 con 0x7f04a8108a50 2024-04-21T10:29:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.726+0000 7f049effd700 1 -- 172.21.15.26:0/704116605 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a0007d00 con 0x7f04a8108a50 2024-04-21T10:29:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.726+0000 7f049effd700 1 -- 172.21.15.26:0/704116605 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a001d450 con 0x7f04a8108a50 2024-04-21T10:29:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.726+0000 7f04aff88700 1 -- 172.21.15.26:0/704116605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a81a5b40 con 0x7f04a8108a50 2024-04-21T10:29:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.727+0000 7f04aff88700 1 -- 172.21.15.26:0/704116605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a8069730 con 0x7f04a8108a50 2024-04-21T10:29:13.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.729+0000 7f049effd700 1 -- 172.21.15.26:0/704116605 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f04a0004000 con 0x7f04a8108a50 2024-04-21T10:29:13.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.729+0000 7f049effd700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f04940a3cd0 0x7f04940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:13.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.729+0000 7f04add24700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f04940a3cd0 0x7f04940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:13.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.729+0000 7f049effd700 1 -- 172.21.15.26:0/704116605 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f04a000f020 con 0x7f04a8108a50 2024-04-21T10:29:13.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.730+0000 7f04add24700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f04940a3cd0 0x7f04940a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f04a4005fd0 tx=0x7f04a4005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:13.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:13.733+0000 7f049effd700 1 -- 172.21.15.26:0/704116605 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04a002a070 con 0x7f04a8108a50 2024-04-21T10:29:14.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.001+0000 7f04aff88700 1 -- 172.21.15.26:0/704116605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f04a802df80 con 0x7f04a8108a50 2024-04-21T10:29:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.006+0000 7f049effd700 1 -- 172.21.15.26:0/704116605 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264992 (secure 0 0 0) 0x7f04a009e740 con 0x7f04a8108a50 2024-04-21T10:29:14.009 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:29:14.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.010+0000 7f04aff88700 1 -- 172.21.15.26:0/704116605 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f04940a3cd0 msgr2=0x7f04940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:14.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.010+0000 7f04aff88700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f04940a3cd0 0x7f04940a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f04a4005fd0 tx=0x7f04a4005d40 comp rx=0 tx=0).stop 2024-04-21T10:29:14.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.010+0000 7f04aff88700 1 -- 172.21.15.26:0/704116605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a8108a50 msgr2=0x7f04a819d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:14.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.010+0000 7f04aff88700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a8108a50 0x7f04a819d890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f04a00062a0 tx=0x7f04a0004ed0 comp rx=0 tx=0).stop 2024-04-21T10:29:14.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.010+0000 7f04aff88700 1 -- 172.21.15.26:0/704116605 shutdown_connections 2024-04-21T10:29:14.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.010+0000 7f04aff88700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f04940a3cd0 0x7f04940a6190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:14.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.010+0000 7f04aff88700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a8108a50 0x7f04a819d890 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:14.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.010+0000 7f04aff88700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a8102a90 0x7f04a8199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.010+0000 7f04aff88700 1 --2- 172.21.15.26:0/704116605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a8103450 0x7f04a8199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.010+0000 7f04aff88700 1 -- 172.21.15.26:0/704116605 >> 172.21.15.26:0/704116605 conn(0x7f04a8079ba0 msgr2=0x7f04a81016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.010+0000 7f04aff88700 1 -- 172.21.15.26:0/704116605 shutdown_connections 2024-04-21T10:29:14.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:14.010+0000 7f04aff88700 1 -- 172.21.15.26:0/704116605 wait complete. 2024-04-21T10:29:14.249 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:13 smithi169 ceph-mon[31469]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.3 KiB/s wr, 19 op/s 2024-04-21T10:29:14.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:13 smithi190 ceph-mon[31423]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.3 KiB/s wr, 19 op/s 2024-04-21T10:29:14.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:13 smithi026 ceph-mon[27965]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.3 KiB/s wr, 19 op/s 2024-04-21T10:29:14.785 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":72610606},{"type":"v1","addr":"172.21.15.26:6801","nonce":72610606}]},"active_addr":"172.21.15.26:6801/72610606","active_change":"2024-04-21T10:22:02.083270+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.26:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":2797499963}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3802401516}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":4245538524}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3125048023}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":2879444176}]}]} 2024-04-21T10:29:14.792 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-21T10:29:14.792 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-21T10:29:14.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T10:29:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:15.163 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:14 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/704116605' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T10:29:15.261 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:14 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/704116605' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T10:29:15.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:14 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/704116605' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T10:29:16.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:15 smithi026 ceph-mon[27965]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.8 KiB/s wr, 17 op/s 2024-04-21T10:29:16.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:15 smithi190 ceph-mon[31423]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.8 KiB/s wr, 17 op/s 2024-04-21T10:29:16.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:15 smithi169 ceph-mon[31469]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.8 KiB/s wr, 17 op/s 2024-04-21T10:29:16.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.790+0000 7ff9369a1700 1 -- 172.21.15.26:0/96510892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff930075590 msgr2=0x7ff930113700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:16.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.790+0000 7ff9369a1700 1 --2- 172.21.15.26:0/96510892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff930075590 0x7ff930113700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff918009990 tx=0x7ff918009ca0 comp rx=0 tx=0).stop 2024-04-21T10:29:16.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.790+0000 7ff9369a1700 1 -- 172.21.15.26:0/96510892 shutdown_connections 2024-04-21T10:29:16.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.790+0000 7ff9369a1700 1 --2- 172.21.15.26:0/96510892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff930113c40 0x7ff930116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:16.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.790+0000 7ff9369a1700 1 --2- 172.21.15.26:0/96510892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff930074c70 0x7ff930075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:16.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.790+0000 7ff9369a1700 1 --2- 172.21.15.26:0/96510892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff930075590 0x7ff930113700 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:16.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.790+0000 7ff9369a1700 1 -- 172.21.15.26:0/96510892 >> 172.21.15.26:0/96510892 conn(0x7ff930079bb0 msgr2=0x7ff930079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:16.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.790+0000 7ff9369a1700 1 -- 172.21.15.26:0/96510892 shutdown_connections 2024-04-21T10:29:16.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.790+0000 7ff9369a1700 1 -- 172.21.15.26:0/96510892 wait complete. 2024-04-21T10:29:16.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff9369a1700 1 Processor -- start 2024-04-21T10:29:16.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff9369a1700 1 -- start start 2024-04-21T10:29:16.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff9369a1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff930074c70 0x7ff9301a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:16.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff9369a1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff930075590 0x7ff9301a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:16.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff9369a1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff930113c40 0x7ff93019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:16.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff9369a1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9301192a0 con 0x7ff930075590 2024-04-21T10:29:16.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff9369a1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff930119120 con 0x7ff930113c40 2024-04-21T10:29:16.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff9369a1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff930119420 con 0x7ff930074c70 2024-04-21T10:29:16.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff92f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff930075590 0x7ff9301a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:16.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff92f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff930075590 0x7ff9301a5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33924/0 (socket says 172.21.15.26:33924) 2024-04-21T10:29:16.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff92f7fe700 1 -- 172.21.15.26:0/2434772753 learned_addr learned my addr 172.21.15.26:0/2434772753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:16.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff934f3e700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff930113c40 0x7ff93019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff92ffff700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff930074c70 0x7ff9301a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff92f7fe700 1 -- 172.21.15.26:0/2434772753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff930074c70 msgr2=0x7ff9301a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff92f7fe700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff930074c70 0x7ff9301a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff92f7fe700 1 -- 172.21.15.26:0/2434772753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff930113c40 msgr2=0x7ff93019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff92f7fe700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff930113c40 0x7ff93019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.791+0000 7ff92f7fe700 1 -- 172.21.15.26:0/2434772753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93019ff40 con 0x7ff930075590 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.792+0000 7ff92ffff700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff930074c70 0x7ff9301a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.792+0000 7ff934f3e700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff930113c40 0x7ff93019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.792+0000 7ff92f7fe700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff930075590 0x7ff9301a5ad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff918000c00 tx=0x7ff9180047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.792+0000 7ff92d7fa700 1 -- 172.21.15.26:0/2434772753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918027070 con 0x7ff930075590 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.792+0000 7ff92d7fa700 1 -- 172.21.15.26:0/2434772753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff918003fa0 con 0x7ff930075590 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.792+0000 7ff9369a1700 1 -- 172.21.15.26:0/2434772753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9301a01d0 con 0x7ff930075590 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.792+0000 7ff92d7fa700 1 -- 172.21.15.26:0/2434772753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918022db0 con 0x7ff930075590 2024-04-21T10:29:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.792+0000 7ff9369a1700 1 -- 172.21.15.26:0/2434772753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9301ac340 con 0x7ff930075590 2024-04-21T10:29:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.793+0000 7ff9369a1700 1 -- 172.21.15.26:0/2434772753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff930076360 con 0x7ff930075590 2024-04-21T10:29:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.794+0000 7ff92d7fa700 1 -- 172.21.15.26:0/2434772753 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff91802a070 con 0x7ff930075590 2024-04-21T10:29:16.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.795+0000 7ff92d7fa700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff91c0a3cd0 0x7ff91c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:16.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.795+0000 7ff92ffff700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff91c0a3cd0 0x7ff91c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:16.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.795+0000 7ff92d7fa700 1 -- 172.21.15.26:0/2434772753 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7ff9180d1470 con 0x7ff930075590 2024-04-21T10:29:16.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.795+0000 7ff92ffff700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff91c0a3cd0 0x7ff91c0a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff920007900 tx=0x7ff920008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:16.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:16.799+0000 7ff92d7fa700 1 -- 172.21.15.26:0/2434772753 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff918013070 con 0x7ff930075590 2024-04-21T10:29:17.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.012+0000 7ff9369a1700 1 -- 172.21.15.26:0/2434772753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9301ac510 con 0x7ff930075590 2024-04-21T10:29:17.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.014+0000 7ff92d7fa700 1 -- 172.21.15.26:0/2434772753 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22207 (secure 0 0 0) 0x7ff91809df30 con 0x7ff930075590 2024-04-21T10:29:17.014 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:29:17.014 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:29:02.925179+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:29:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.016+0000 7ff9369a1700 1 -- 172.21.15.26:0/2434772753 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff91c0a3cd0 msgr2=0x7ff91c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.016+0000 7ff9369a1700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff91c0a3cd0 0x7ff91c0a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff920007900 tx=0x7ff920008040 comp rx=0 tx=0).stop 2024-04-21T10:29:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.016+0000 7ff9369a1700 1 -- 172.21.15.26:0/2434772753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff930075590 msgr2=0x7ff9301a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.016+0000 7ff9369a1700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff930075590 0x7ff9301a5ad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff918000c00 tx=0x7ff9180047b0 comp rx=0 tx=0).stop 2024-04-21T10:29:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.016+0000 7ff9369a1700 1 -- 172.21.15.26:0/2434772753 shutdown_connections 2024-04-21T10:29:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.016+0000 7ff9369a1700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff91c0a3cd0 0x7ff91c0a6190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.016+0000 7ff9369a1700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff930113c40 0x7ff93019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.016+0000 7ff9369a1700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff930074c70 0x7ff9301a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.016+0000 7ff9369a1700 1 --2- 172.21.15.26:0/2434772753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff930075590 0x7ff9301a5ad0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.016+0000 7ff9369a1700 1 -- 172.21.15.26:0/2434772753 >> 172.21.15.26:0/2434772753 conn(0x7ff930079bb0 msgr2=0x7ff930112ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.016+0000 7ff9369a1700 1 -- 172.21.15.26:0/2434772753 shutdown_connections 2024-04-21T10:29:17.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:17.016+0000 7ff9369a1700 1 -- 172.21.15.26:0/2434772753 wait complete. 2024-04-21T10:29:17.641 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-21T10:29:17.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T10:29:17.898 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:18.275 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:17 smithi026 ceph-mon[27965]: pgmap v294: 97 pgs: 97 active+clean; 623 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.3 KiB/s wr, 16 op/s 2024-04-21T10:29:18.275 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:17 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2434772753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:18.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:17 smithi190 ceph-mon[31423]: pgmap v294: 97 pgs: 97 active+clean; 623 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.3 KiB/s wr, 16 op/s 2024-04-21T10:29:18.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:17 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2434772753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:18.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:17 smithi169 ceph-mon[31469]: pgmap v294: 97 pgs: 97 active+clean; 623 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.3 KiB/s wr, 16 op/s 2024-04-21T10:29:18.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:17 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2434772753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:19.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.558+0000 7facc42d6700 1 -- 172.21.15.26:0/2019687627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facbc102ac0 msgr2=0x7facbc102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:19.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.558+0000 7facc42d6700 1 --2- 172.21.15.26:0/2019687627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facbc102ac0 0x7facbc102f40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7facac009990 tx=0x7facac009ca0 comp rx=0 tx=0).stop 2024-04-21T10:29:19.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.558+0000 7facc42d6700 1 -- 172.21.15.26:0/2019687627 shutdown_connections 2024-04-21T10:29:19.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.558+0000 7facc42d6700 1 --2- 172.21.15.26:0/2019687627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facbc108ab0 0x7facbc108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:19.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.558+0000 7facc42d6700 1 --2- 172.21.15.26:0/2019687627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facbc103480 0x7facbc10f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.558+0000 7facc42d6700 1 --2- 172.21.15.26:0/2019687627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facbc102ac0 0x7facbc102f40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.558+0000 7facc42d6700 1 -- 172.21.15.26:0/2019687627 >> 172.21.15.26:0/2019687627 conn(0x7facbc079bf0 msgr2=0x7facbc07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.558+0000 7facc42d6700 1 -- 172.21.15.26:0/2019687627 shutdown_connections 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.558+0000 7facc42d6700 1 -- 172.21.15.26:0/2019687627 wait complete. 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc42d6700 1 Processor -- start 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc42d6700 1 -- start start 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc42d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facbc102ac0 0x7facbc1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc42d6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facbc103480 0x7facbc199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc42d6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facbc108ab0 0x7facbc19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc42d6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facbc112cf0 con 0x7facbc102ac0 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc42d6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7facbc112b70 con 0x7facbc108ab0 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc42d6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7facbc112e70 con 0x7facbc103480 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc2072700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facbc102ac0 0x7facbc1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:19.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc2072700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facbc102ac0 0x7facbc1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33946/0 (socket says 172.21.15.26:33946) 2024-04-21T10:29:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc2072700 1 -- 172.21.15.26:0/1334546088 learned_addr learned my addr 172.21.15.26:0/1334546088 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc2873700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facbc108ab0 0x7facbc19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.559+0000 7facc1871700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facbc103480 0x7facbc199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.560+0000 7facc2072700 1 -- 172.21.15.26:0/1334546088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facbc103480 msgr2=0x7facbc199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.560+0000 7facc2072700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facbc103480 0x7facbc199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.560+0000 7facc2072700 1 -- 172.21.15.26:0/1334546088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facbc108ab0 msgr2=0x7facbc19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.560+0000 7facc2072700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facbc108ab0 0x7facbc19dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.560+0000 7facc2072700 1 -- 172.21.15.26:0/1334546088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facbc19e1f0 con 0x7facbc102ac0 2024-04-21T10:29:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.560+0000 7facc2072700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facbc102ac0 0x7facbc1991e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7facb800b530 tx=0x7facb800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.560+0000 7facb37fe700 1 -- 172.21.15.26:0/1334546088 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8015020 con 0x7facbc102ac0 2024-04-21T10:29:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.560+0000 7facc42d6700 1 -- 172.21.15.26:0/1334546088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facbc19e4e0 con 0x7facbc102ac0 2024-04-21T10:29:19.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.560+0000 7facb37fe700 1 -- 172.21.15.26:0/1334546088 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb8011ce0 con 0x7facbc102ac0 2024-04-21T10:29:19.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.560+0000 7facb37fe700 1 -- 172.21.15.26:0/1334546088 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8010dd0 con 0x7facbc102ac0 2024-04-21T10:29:19.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.560+0000 7facc42d6700 1 -- 172.21.15.26:0/1334546088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facbc1a5d60 con 0x7facbc102ac0 2024-04-21T10:29:19.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.561+0000 7facc42d6700 1 -- 172.21.15.26:0/1334546088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facbc1041f0 con 0x7facbc102ac0 2024-04-21T10:29:19.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.563+0000 7facb37fe700 1 -- 172.21.15.26:0/1334546088 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7facb8018020 con 0x7facbc102ac0 2024-04-21T10:29:19.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.564+0000 7facb37fe700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7faca80a3cd0 0x7faca80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:19.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.564+0000 7facc1871700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7faca80a3cd0 0x7faca80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:19.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.564+0000 7facb37fe700 1 -- 172.21.15.26:0/1334546088 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7facb80cffe0 con 0x7facbc102ac0 2024-04-21T10:29:19.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.564+0000 7facc1871700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7faca80a3cd0 0x7faca80a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7facac00b5c0 tx=0x7facac005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:19.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.568+0000 7facb37fe700 1 -- 172.21.15.26:0/1334546088 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7facb801f020 con 0x7facbc102ac0 2024-04-21T10:29:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.777+0000 7facc42d6700 1 -- 172.21.15.26:0/1334546088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facbc19a540 con 0x7facbc102ac0 2024-04-21T10:29:19.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.779+0000 7facb37fe700 1 -- 172.21.15.26:0/1334546088 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22207 (secure 0 0 0) 0x7facb809ca40 con 0x7facbc102ac0 2024-04-21T10:29:19.779 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:29:19.779 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:29:02.925179+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:29:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.781+0000 7facc42d6700 1 -- 172.21.15.26:0/1334546088 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7faca80a3cd0 msgr2=0x7faca80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.781+0000 7facc42d6700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7faca80a3cd0 0x7faca80a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7facac00b5c0 tx=0x7facac005960 comp rx=0 tx=0).stop 2024-04-21T10:29:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.781+0000 7facc42d6700 1 -- 172.21.15.26:0/1334546088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facbc102ac0 msgr2=0x7facbc1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.781+0000 7facc42d6700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facbc102ac0 0x7facbc1991e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7facb800b530 tx=0x7facb800b840 comp rx=0 tx=0).stop 2024-04-21T10:29:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.781+0000 7facc42d6700 1 -- 172.21.15.26:0/1334546088 shutdown_connections 2024-04-21T10:29:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.781+0000 7facc42d6700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7faca80a3cd0 0x7faca80a6190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.781+0000 7facc42d6700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facbc108ab0 0x7facbc19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.781+0000 7facc42d6700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facbc102ac0 0x7facbc1991e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.781+0000 7facc42d6700 1 --2- 172.21.15.26:0/1334546088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facbc103480 0x7facbc199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.781+0000 7facc42d6700 1 -- 172.21.15.26:0/1334546088 >> 172.21.15.26:0/1334546088 conn(0x7facbc079bf0 msgr2=0x7facbc100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.781+0000 7facc42d6700 1 -- 172.21.15.26:0/1334546088 shutdown_connections 2024-04-21T10:29:19.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:19.781+0000 7facc42d6700 1 -- 172.21.15.26:0/1334546088 wait complete. 2024-04-21T10:29:20.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:19 smithi190 ceph-mon[31423]: pgmap v295: 97 pgs: 97 active+clean; 631 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-04-21T10:29:20.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:19 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1334546088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:20.378 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:19 smithi026 ceph-mon[27965]: pgmap v295: 97 pgs: 97 active+clean; 631 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-04-21T10:29:20.378 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:19 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1334546088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:20.408 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-21T10:29:20.408 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-21T10:29:20.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-21T10:29:20.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-21T10:29:20.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-21T10:29:20.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-21T10:29:20.411 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-21T10:29:20.411 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-21T10:29:20.411 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-21T10:29:20.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-21T10:29:20.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-21T10:29:20.413 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-21T10:29:20.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:19 smithi169 ceph-mon[31469]: pgmap v295: 97 pgs: 97 active+clean; 631 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-04-21T10:29:20.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:19 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1334546088' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:29:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:21.168 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:21.168 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:21.172 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:21.266 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:22.273 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:21 smithi026 ceph-mon[27965]: pgmap v296: 97 pgs: 97 active+clean; 631 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-21T10:29:22.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:21 smithi190 ceph-mon[31423]: pgmap v296: 97 pgs: 97 active+clean; 631 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-21T10:29:22.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:21 smithi169 ceph-mon[31469]: pgmap v296: 97 pgs: 97 active+clean; 631 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-21T10:29:24.118 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:23 smithi026 ceph-mon[27965]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-04-21T10:29:24.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:23 smithi190 ceph-mon[31423]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-04-21T10:29:24.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:23 smithi169 ceph-mon[31469]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-04-21T10:29:26.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:25 smithi190 ceph-mon[31423]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-21T10:29:26.306 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:25 smithi026 ceph-mon[27965]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-21T10:29:26.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:25 smithi169 ceph-mon[31469]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-21T10:29:28.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:27 smithi190 ceph-mon[31423]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-21T10:29:28.286 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:27 smithi026 ceph-mon[27965]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-21T10:29:28.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:27 smithi169 ceph-mon[31469]: pgmap v299: 97 pgs: 97 active+clean; 635 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-21T10:29:30.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:29 smithi026 ceph-mon[27965]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-21T10:29:30.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:29 smithi190 ceph-mon[31423]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-21T10:29:30.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:29 smithi169 ceph-mon[31469]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-21T10:29:30.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.558+0000 7f32ff096700 1 -- 172.21.15.26:0/3289401832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8103480 msgr2=0x7f32f810f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:30.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.558+0000 7f32ff096700 1 --2- 172.21.15.26:0/3289401832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8103480 0x7f32f810f8d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b0a0 tx=0x7f32f000b3b0 comp rx=0 tx=0).stop 2024-04-21T10:29:30.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.558+0000 7f32ff096700 1 -- 172.21.15.26:0/3289401832 shutdown_connections 2024-04-21T10:29:30.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.558+0000 7f32ff096700 1 --2- 172.21.15.26:0/3289401832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8103480 0x7f32f810f8d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:30.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.558+0000 7f32ff096700 1 --2- 172.21.15.26:0/3289401832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32f8108a70 0x7f32f8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:30.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.558+0000 7f32ff096700 1 --2- 172.21.15.26:0/3289401832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f8102ac0 0x7f32f8102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:30.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.558+0000 7f32ff096700 1 -- 172.21.15.26:0/3289401832 >> 172.21.15.26:0/3289401832 conn(0x7f32f8079bb0 msgr2=0x7f32f8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:30.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.558+0000 7f32ff096700 1 -- 172.21.15.26:0/3289401832 shutdown_connections 2024-04-21T10:29:30.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.558+0000 7f32ff096700 1 -- 172.21.15.26:0/3289401832 wait complete. 2024-04-21T10:29:30.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.559+0000 7f32ff096700 1 Processor -- start 2024-04-21T10:29:30.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.559+0000 7f32ff096700 1 -- start start 2024-04-21T10:29:30.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.559+0000 7f32ff096700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f8102ac0 0x7f32f8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.559+0000 7f32ff096700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32f8103480 0x7f32f8199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.559+0000 7f32ff096700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8108a70 0x7f32f819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.559+0000 7f32ff096700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8112c20 con 0x7f32f8108a70 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.559+0000 7f32ff096700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8112aa0 con 0x7f32f8102ac0 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.559+0000 7f32ff096700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8112da0 con 0x7f32f8103480 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.559+0000 7f32fd633700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8108a70 0x7f32f819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.559+0000 7f32fd633700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8108a70 0x7f32f819d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42332/0 (socket says 172.21.15.26:42332) 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.559+0000 7f32fd633700 1 -- 172.21.15.26:0/2260119139 learned_addr learned my addr 172.21.15.26:0/2260119139 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.559+0000 7f32fce32700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f8102ac0 0x7f32f8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32f7fff700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32f8103480 0x7f32f8199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32fd633700 1 -- 172.21.15.26:0/2260119139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32f8103480 msgr2=0x7f32f8199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32fd633700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32f8103480 0x7f32f8199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32fd633700 1 -- 172.21.15.26:0/2260119139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f8102ac0 msgr2=0x7f32f8199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32fd633700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f8102ac0 0x7f32f8199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:30.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32fd633700 1 -- 172.21.15.26:0/2260119139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f819e120 con 0x7f32f8108a70 2024-04-21T10:29:30.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32f7fff700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32f8103480 0x7f32f8199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:29:30.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32fce32700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f8102ac0 0x7f32f8199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:29:30.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32fd633700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8108a70 0x7f32f819d9e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f32f00062a0 tx=0x7f32f0004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:30.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32f5ffb700 1 -- 172.21.15.26:0/2260119139 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0021070 con 0x7f32f8108a70 2024-04-21T10:29:30.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32f5ffb700 1 -- 172.21.15.26:0/2260119139 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f0003c00 con 0x7f32f8108a70 2024-04-21T10:29:30.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32ff096700 1 -- 172.21.15.26:0/2260119139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f819e3b0 con 0x7f32f8108a70 2024-04-21T10:29:30.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32f5ffb700 1 -- 172.21.15.26:0/2260119139 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0004830 con 0x7f32f8108a70 2024-04-21T10:29:30.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.560+0000 7f32ff096700 1 -- 172.21.15.26:0/2260119139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f81a5c90 con 0x7f32f8108a70 2024-04-21T10:29:30.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.561+0000 7f32ff096700 1 -- 172.21.15.26:0/2260119139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32f8063150 con 0x7f32f8108a70 2024-04-21T10:29:30.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.563+0000 7f32f5ffb700 1 -- 172.21.15.26:0/2260119139 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f32f0007a00 con 0x7f32f8108a70 2024-04-21T10:29:30.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.563+0000 7f32f5ffb700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f32e80a3cd0 0x7f32e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:30.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.563+0000 7f32f5ffb700 1 -- 172.21.15.26:0/2260119139 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f32f000f020 con 0x7f32f8108a70 2024-04-21T10:29:30.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.563+0000 7f32fce32700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f32e80a3cd0 0x7f32e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:30.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.564+0000 7f32f5ffb700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185] conn(0x7f32e80ad4b0 0x7f32e80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:30.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.564+0000 7f32f5ffb700 1 -- 172.21.15.26:0/2260119139 --> [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32f0003e00 con 0x7f32e80ad4b0 2024-04-21T10:29:30.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.564+0000 7f32f5ffb700 1 -- 172.21.15.26:0/2260119139 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f32f00d2010 con 0x7f32f8108a70 2024-04-21T10:29:30.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.564+0000 7f32fce32700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f32e80a3cd0 0x7f32e80a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009c90 tx=0x7f32ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:30.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.564+0000 7f32f7fff700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185] conn(0x7f32e80ad4b0 0x7f32e80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:30.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.564+0000 7f32f7fff700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185] conn(0x7f32e80ad4b0 0x7f32e80af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:30.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.565+0000 7f32f5ffb700 1 -- 172.21.15.26:0/2260119139 <== osd.7 v2:172.21.15.169:6816/16938185 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32f0003e00 con 0x7f32e80ad4b0 2024-04-21T10:29:30.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.582+0000 7f32ff096700 1 -- 172.21.15.26:0/2260119139 --> [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32f80635d0 con 0x7f32e80ad4b0 2024-04-21T10:29:30.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.582+0000 7f32f5ffb700 1 -- 172.21.15.26:0/2260119139 <== osd.7 v2:172.21.15.169:6816/16938185 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f32f80635d0 con 0x7f32e80ad4b0 2024-04-21T10:29:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.583+0000 7f32ff096700 1 -- 172.21.15.26:0/2260119139 >> [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185] conn(0x7f32e80ad4b0 msgr2=0x7f32e80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.583+0000 7f32ff096700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185] conn(0x7f32e80ad4b0 0x7f32e80af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.583+0000 7f32ff096700 1 -- 172.21.15.26:0/2260119139 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f32e80a3cd0 msgr2=0x7f32e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.583+0000 7f32ff096700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f32e80a3cd0 0x7f32e80a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009c90 tx=0x7f32ec009260 comp rx=0 tx=0).stop 2024-04-21T10:29:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.583+0000 7f32ff096700 1 -- 172.21.15.26:0/2260119139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8108a70 msgr2=0x7f32f819d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.583+0000 7f32ff096700 1 --2- 172.21.15.26:0/2260119139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8108a70 0x7f32f819d9e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f32f00062a0 tx=0x7f32f0004d30 comp rx=0 tx=0).stop 2024-04-21T10:29:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.583+0000 7f32fd633700 1 -- 172.21.15.26:0/2260119139 reap_dead start 2024-04-21T10:29:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.583+0000 7f32ff096700 1 -- 172.21.15.26:0/2260119139 shutdown_connections 2024-04-21T10:29:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.583+0000 7f32ff096700 1 -- 172.21.15.26:0/2260119139 >> 172.21.15.26:0/2260119139 conn(0x7f32f8079bb0 msgr2=0x7f32f8101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.583+0000 7f32ff096700 1 -- 172.21.15.26:0/2260119139 shutdown_connections 2024-04-21T10:29:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:30.583+0000 7f32ff096700 1 -- 172.21.15.26:0/2260119139 wait complete. 2024-04-21T10:29:31.115 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:30 smithi026 ceph-mon[27965]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-21T10:29:31.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:30 smithi190 ceph-mon[31423]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-21T10:29:31.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:30 smithi169 ceph-mon[31469]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-21T10:29:33.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:33 smithi026 ceph-mon[27965]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2024-04-21T10:29:33.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:33 smithi190 ceph-mon[31423]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2024-04-21T10:29:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.159+0000 7f49262bf700 1 -- 172.21.15.26:0/2628619257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4920101d10 msgr2=0x7f4920105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.159+0000 7f49262bf700 1 --2- 172.21.15.26:0/2628619257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4920101d10 0x7f4920105d10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f49080099c0 tx=0x7f4908009cd0 comp rx=0 tx=0).stop 2024-04-21T10:29:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.160+0000 7f49262bf700 1 -- 172.21.15.26:0/2628619257 shutdown_connections 2024-04-21T10:29:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.160+0000 7f49262bf700 1 --2- 172.21.15.26:0/2628619257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49201063f0 0x7f492010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.160+0000 7f49262bf700 1 --2- 172.21.15.26:0/2628619257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4920101360 0x7f4920101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.160+0000 7f49262bf700 1 --2- 172.21.15.26:0/2628619257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4920101d10 0x7f4920105d10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.160+0000 7f49262bf700 1 -- 172.21.15.26:0/2628619257 >> 172.21.15.26:0/2628619257 conn(0x7f4920077fb0 msgr2=0x7f49200ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.160+0000 7f49262bf700 1 -- 172.21.15.26:0/2628619257 shutdown_connections 2024-04-21T10:29:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.160+0000 7f49262bf700 1 -- 172.21.15.26:0/2628619257 wait complete. 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.160+0000 7f49262bf700 1 Processor -- start 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.160+0000 7f49262bf700 1 -- start start 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.161+0000 7f49262bf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4920101360 0x7f4920197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.161+0000 7f49262bf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4920101d10 0x7f49201982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.161+0000 7f49262bf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49201063f0 0x7f49201987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.161+0000 7f49262bf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4920111af0 con 0x7f4920101d10 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.161+0000 7f49262bf700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4920111970 con 0x7f49201063f0 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.161+0000 7f49262bf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4920111c70 con 0x7f4920101360 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.161+0000 7f4917fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4920101d10 0x7f49201982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.161+0000 7f4917fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4920101d10 0x7f49201982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42356/0 (socket says 172.21.15.26:42356) 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.161+0000 7f4917fff700 1 -- 172.21.15.26:0/2270965318 learned_addr learned my addr 172.21.15.26:0/2270965318 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.161+0000 7f491ffff700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4920101360 0x7f4920197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.161+0000 7f492485c700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49201063f0 0x7f49201987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.162+0000 7f4917fff700 1 -- 172.21.15.26:0/2270965318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4920101360 msgr2=0x7f4920197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.162+0000 7f4917fff700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4920101360 0x7f4920197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.162+0000 7f4917fff700 1 -- 172.21.15.26:0/2270965318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49201063f0 msgr2=0x7f49201987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.162+0000 7f4917fff700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49201063f0 0x7f49201987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.162+0000 7f4917fff700 1 -- 172.21.15.26:0/2270965318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49201a75d0 con 0x7f4920101d10 2024-04-21T10:29:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.162+0000 7f4917fff700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4920101d10 0x7f49201982a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4908006010 tx=0x7f49080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.162+0000 7f491dffb700 1 -- 172.21.15.26:0/2270965318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908027070 con 0x7f4920101d10 2024-04-21T10:29:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.162+0000 7f49262bf700 1 -- 172.21.15.26:0/2270965318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49201a7860 con 0x7f4920101d10 2024-04-21T10:29:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.162+0000 7f491dffb700 1 -- 172.21.15.26:0/2270965318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4908003f20 con 0x7f4920101d10 2024-04-21T10:29:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.162+0000 7f491dffb700 1 -- 172.21.15.26:0/2270965318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908022cd0 con 0x7f4920101d10 2024-04-21T10:29:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.162+0000 7f49262bf700 1 -- 172.21.15.26:0/2270965318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49201a7d10 con 0x7f4920101d10 2024-04-21T10:29:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.163+0000 7f49262bf700 1 -- 172.21.15.26:0/2270965318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49200631d0 con 0x7f4920101d10 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.164+0000 7f491dffb700 1 -- 172.21.15.26:0/2270965318 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f490802a070 con 0x7f4920101d10 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.165+0000 7f491dffb700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f490c0a3cd0 0x7f490c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.165+0000 7f491ffff700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f490c0a3cd0 0x7f490c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.165+0000 7f491dffb700 1 -- 172.21.15.26:0/2270965318 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f49080d1c80 con 0x7f4920101d10 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.166+0000 7f491dffb700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945] conn(0x7f490c0ad4b0 0x7f490c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.166+0000 7f491dffb700 1 -- 172.21.15.26:0/2270965318 --> [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4908010ea0 con 0x7f490c0ad4b0 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.166+0000 7f491dffb700 1 -- 172.21.15.26:0/2270965318 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f49080d2030 con 0x7f4920101d10 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.166+0000 7f491ffff700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f490c0a3cd0 0x7f490c0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4910005950 tx=0x7f491000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.166+0000 7f492485c700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945] conn(0x7f490c0ad4b0 0x7f490c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.167+0000 7f492485c700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945] conn(0x7f490c0ad4b0 0x7f490c0af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.167+0000 7f491dffb700 1 -- 172.21.15.26:0/2270965318 <== osd.10 v2:172.21.15.169:6824/3439107945 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4908010ea0 con 0x7f490c0ad4b0 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.202+0000 7f49262bf700 1 -- 172.21.15.26:0/2270965318 --> [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4920063650 con 0x7f490c0ad4b0 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.203+0000 7f491dffb700 1 -- 172.21.15.26:0/2270965318 <== osd.10 v2:172.21.15.169:6824/3439107945 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4920063650 con 0x7f490c0ad4b0 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.203+0000 7f49262bf700 1 -- 172.21.15.26:0/2270965318 >> [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945] conn(0x7f490c0ad4b0 msgr2=0x7f490c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.203+0000 7f49262bf700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945] conn(0x7f490c0ad4b0 0x7f490c0af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.203+0000 7f49262bf700 1 -- 172.21.15.26:0/2270965318 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f490c0a3cd0 msgr2=0x7f490c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.203+0000 7f49262bf700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f490c0a3cd0 0x7f490c0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4910005950 tx=0x7f491000b410 comp rx=0 tx=0).stop 2024-04-21T10:29:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.203+0000 7f49262bf700 1 -- 172.21.15.26:0/2270965318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4920101d10 msgr2=0x7f49201982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.203+0000 7f49262bf700 1 --2- 172.21.15.26:0/2270965318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4920101d10 0x7f49201982a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4908006010 tx=0x7f49080047b0 comp rx=0 tx=0).stop 2024-04-21T10:29:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.203+0000 7f492485c700 1 -- 172.21.15.26:0/2270965318 reap_dead start 2024-04-21T10:29:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.203+0000 7f49262bf700 1 -- 172.21.15.26:0/2270965318 shutdown_connections 2024-04-21T10:29:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.203+0000 7f49262bf700 1 -- 172.21.15.26:0/2270965318 >> 172.21.15.26:0/2270965318 conn(0x7f4920077fb0 msgr2=0x7f49200ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.203+0000 7f49262bf700 1 -- 172.21.15.26:0/2270965318 shutdown_connections 2024-04-21T10:29:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.203+0000 7f49262bf700 1 -- 172.21.15.26:0/2270965318 wait complete. 2024-04-21T10:29:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:33 smithi169 ceph-mon[31469]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2024-04-21T10:29:34.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.279+0000 7fe6f078e700 1 -- 172.21.15.26:0/57143782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6e81034b0 msgr2=0x7fe6e810f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.279+0000 7fe6f078e700 1 --2- 172.21.15.26:0/57143782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6e81034b0 0x7fe6e810f8e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009990 tx=0x7fe6e0009ca0 comp rx=0 tx=0).stop 2024-04-21T10:29:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.279+0000 7fe6f078e700 1 -- 172.21.15.26:0/57143782 shutdown_connections 2024-04-21T10:29:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.279+0000 7fe6f078e700 1 --2- 172.21.15.26:0/57143782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e8108a80 0x7fe6e8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.279+0000 7fe6f078e700 1 --2- 172.21.15.26:0/57143782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6e81034b0 0x7fe6e810f8e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.279+0000 7fe6f078e700 1 --2- 172.21.15.26:0/57143782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6e8102af0 0x7fe6e8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.279+0000 7fe6f078e700 1 -- 172.21.15.26:0/57143782 >> 172.21.15.26:0/57143782 conn(0x7fe6e8079b20 msgr2=0x7fe6e8079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.279+0000 7fe6f078e700 1 -- 172.21.15.26:0/57143782 shutdown_connections 2024-04-21T10:29:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.279+0000 7fe6f078e700 1 -- 172.21.15.26:0/57143782 wait complete. 2024-04-21T10:29:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6f078e700 1 Processor -- start 2024-04-21T10:29:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6f078e700 1 -- start start 2024-04-21T10:29:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6f078e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e8102af0 0x7fe6e8199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6f078e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6e81034b0 0x7fe6e8199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6f078e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6e8108a80 0x7fe6e819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6f078e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8112c30 con 0x7fe6e81034b0 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6f078e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8112ab0 con 0x7fe6e8102af0 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6f078e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8112db0 con 0x7fe6e8108a80 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6edd29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6e81034b0 0x7fe6e8199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6edd29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6e81034b0 0x7fe6e8199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42372/0 (socket says 172.21.15.26:42372) 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6edd29700 1 -- 172.21.15.26:0/2643720380 learned_addr learned my addr 172.21.15.26:0/2643720380 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6ee52a700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e8102af0 0x7fe6e8199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6edd29700 1 -- 172.21.15.26:0/2643720380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6e8108a80 msgr2=0x7fe6e819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6edd29700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6e8108a80 0x7fe6e819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6edd29700 1 -- 172.21.15.26:0/2643720380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e8102af0 msgr2=0x7fe6e8199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6edd29700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e8102af0 0x7fe6e8199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6edd29700 1 -- 172.21.15.26:0/2643720380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e819e030 con 0x7fe6e81034b0 2024-04-21T10:29:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.280+0000 7fe6ee52a700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e8102af0 0x7fe6e8199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.281+0000 7fe6edd29700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6e81034b0 0x7fe6e8199560 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00c9b0 tx=0x7fe6dc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.281+0000 7fe6e77fe700 1 -- 172.21.15.26:0/2643720380 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc007b00 con 0x7fe6e81034b0 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.281+0000 7fe6e77fe700 1 -- 172.21.15.26:0/2643720380 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6dc007ca0 con 0x7fe6e81034b0 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.281+0000 7fe6f078e700 1 -- 172.21.15.26:0/2643720380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e819e320 con 0x7fe6e81034b0 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.281+0000 7fe6e77fe700 1 -- 172.21.15.26:0/2643720380 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc018730 con 0x7fe6e81034b0 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.281+0000 7fe6f078e700 1 -- 172.21.15.26:0/2643720380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e81a5ba0 con 0x7fe6e81034b0 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.282+0000 7fe6f078e700 1 -- 172.21.15.26:0/2643720380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe6e80631d0 con 0x7fe6e81034b0 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.283+0000 7fe6e77fe700 1 -- 172.21.15.26:0/2643720380 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe6dc00f450 con 0x7fe6e81034b0 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.284+0000 7fe6e77fe700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe6d00a3cd0 0x7fe6d00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.284+0000 7fe6ee52a700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe6d00a3cd0 0x7fe6d00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.284+0000 7fe6e77fe700 1 -- 172.21.15.26:0/2643720380 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fe6dc0d09f0 con 0x7fe6e81034b0 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.284+0000 7fe6e77fe700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642] conn(0x7fe6d00ad4b0 0x7fe6d00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.284+0000 7fe6e77fe700 1 -- 172.21.15.26:0/2643720380 --> [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6dc004500 con 0x7fe6d00ad4b0 2024-04-21T10:29:34.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.284+0000 7fe6e77fe700 1 -- 172.21.15.26:0/2643720380 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fe6dc0d0da0 con 0x7fe6e81034b0 2024-04-21T10:29:34.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.284+0000 7fe6ee52a700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe6d00a3cd0 0x7fe6d00a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009790 tx=0x7fe6d8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:34.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.284+0000 7fe6eed2b700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642] conn(0x7fe6d00ad4b0 0x7fe6d00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:34.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.285+0000 7fe6eed2b700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642] conn(0x7fe6d00ad4b0 0x7fe6d00af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:34.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.286+0000 7fe6e77fe700 1 -- 172.21.15.26:0/2643720380 <== osd.4 v2:172.21.15.169:6808/1708888642 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6dc004500 con 0x7fe6d00ad4b0 2024-04-21T10:29:34.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.302+0000 7fe6f078e700 1 -- 172.21.15.26:0/2643720380 --> [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe6e8063650 con 0x7fe6d00ad4b0 2024-04-21T10:29:34.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.303+0000 7fe6e77fe700 1 -- 172.21.15.26:0/2643720380 <== osd.4 v2:172.21.15.169:6808/1708888642 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe6e8063650 con 0x7fe6d00ad4b0 2024-04-21T10:29:34.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.303+0000 7fe6f078e700 1 -- 172.21.15.26:0/2643720380 >> [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642] conn(0x7fe6d00ad4b0 msgr2=0x7fe6d00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:34.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.303+0000 7fe6f078e700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642] conn(0x7fe6d00ad4b0 0x7fe6d00af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:34.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.303+0000 7fe6f078e700 1 -- 172.21.15.26:0/2643720380 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe6d00a3cd0 msgr2=0x7fe6d00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:34.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.303+0000 7fe6f078e700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe6d00a3cd0 0x7fe6d00a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009790 tx=0x7fe6d8009040 comp rx=0 tx=0).stop 2024-04-21T10:29:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.303+0000 7fe6f078e700 1 -- 172.21.15.26:0/2643720380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6e81034b0 msgr2=0x7fe6e8199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.303+0000 7fe6f078e700 1 --2- 172.21.15.26:0/2643720380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6e81034b0 0x7fe6e8199560 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00c9b0 tx=0x7fe6dc00cd70 comp rx=0 tx=0).stop 2024-04-21T10:29:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.303+0000 7fe6eed2b700 1 -- 172.21.15.26:0/2643720380 reap_dead start 2024-04-21T10:29:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.303+0000 7fe6f078e700 1 -- 172.21.15.26:0/2643720380 shutdown_connections 2024-04-21T10:29:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.303+0000 7fe6f078e700 1 -- 172.21.15.26:0/2643720380 >> 172.21.15.26:0/2643720380 conn(0x7fe6e8079b20 msgr2=0x7fe6e8101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.303+0000 7fe6f078e700 1 -- 172.21.15.26:0/2643720380 shutdown_connections 2024-04-21T10:29:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:33.303+0000 7fe6f078e700 1 -- 172.21.15.26:0/2643720380 wait complete. 2024-04-21T10:29:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.591+0000 7feabe273700 1 -- 172.21.15.26:0/1901599611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab8102a90 msgr2=0x7feab8102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.591+0000 7feabe273700 1 --2- 172.21.15.26:0/1901599611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab8102a90 0x7feab8102f10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feaac009990 tx=0x7feaac009ca0 comp rx=0 tx=0).stop 2024-04-21T10:29:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.592+0000 7feabe273700 1 -- 172.21.15.26:0/1901599611 shutdown_connections 2024-04-21T10:29:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.592+0000 7feabe273700 1 --2- 172.21.15.26:0/1901599611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feab8103450 0x7feab810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:35.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.592+0000 7feabe273700 1 --2- 172.21.15.26:0/1901599611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab8108a50 0x7feab8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.592+0000 7feabe273700 1 --2- 172.21.15.26:0/1901599611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab8102a90 0x7feab8102f10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.592+0000 7feabe273700 1 -- 172.21.15.26:0/1901599611 >> 172.21.15.26:0/1901599611 conn(0x7feab8079ba0 msgr2=0x7feab8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.592+0000 7feabe273700 1 -- 172.21.15.26:0/1901599611 shutdown_connections 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.592+0000 7feabe273700 1 -- 172.21.15.26:0/1901599611 wait complete. 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.592+0000 7feabe273700 1 Processor -- start 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.592+0000 7feabe273700 1 -- start start 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feabe273700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab8102a90 0x7feab81990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feabe273700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab8103450 0x7feab8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feabe273700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feab8108a50 0x7feab819d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feabe273700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feab8112c60 con 0x7feab8102a90 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feabe273700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feab8112ae0 con 0x7feab8108a50 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feabe273700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feab8112de0 con 0x7feab8103450 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feab7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab8102a90 0x7feab81990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feab7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab8102a90 0x7feab81990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54536/0 (socket says 172.21.15.26:54536) 2024-04-21T10:29:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feab7fff700 1 -- 172.21.15.26:0/1001516791 learned_addr learned my addr 172.21.15.26:0/1001516791 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feab77fe700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab8103450 0x7feab8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feabc810700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feab8108a50 0x7feab819d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feab7fff700 1 -- 172.21.15.26:0/1001516791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab8103450 msgr2=0x7feab8199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feab7fff700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab8103450 0x7feab8199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feab7fff700 1 -- 172.21.15.26:0/1001516791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feab8108a50 msgr2=0x7feab819d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feab7fff700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feab8108a50 0x7feab819d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feab7fff700 1 -- 172.21.15.26:0/1001516791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab819e100 con 0x7feab8102a90 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feab77fe700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feab8103450 0x7feab8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.593+0000 7feab7fff700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab8102a90 0x7feab81990f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feaa800b530 tx=0x7feaa800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.594+0000 7feab57fa700 1 -- 172.21.15.26:0/1001516791 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8015020 con 0x7feab8102a90 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.594+0000 7feabe273700 1 -- 172.21.15.26:0/1001516791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab819e3f0 con 0x7feab8102a90 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.594+0000 7feab57fa700 1 -- 172.21.15.26:0/1001516791 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaa8011ce0 con 0x7feab8102a90 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.594+0000 7feab57fa700 1 -- 172.21.15.26:0/1001516791 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8010d20 con 0x7feab8102a90 2024-04-21T10:29:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.594+0000 7feabe273700 1 -- 172.21.15.26:0/1001516791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab81a5c70 con 0x7feab8102a90 2024-04-21T10:29:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.594+0000 7feabe273700 1 -- 172.21.15.26:0/1001516791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feab80631d0 con 0x7feab8102a90 2024-04-21T10:29:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.596+0000 7feab57fa700 1 -- 172.21.15.26:0/1001516791 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feaa8018020 con 0x7feab8102a90 2024-04-21T10:29:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.596+0000 7feab57fa700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7feaa00a3cd0 0x7feaa00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.597+0000 7feab77fe700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7feaa00a3cd0 0x7feaa00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.597+0000 7feab57fa700 1 -- 172.21.15.26:0/1001516791 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7feaa80d0360 con 0x7feab8102a90 2024-04-21T10:29:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.597+0000 7feab57fa700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539] conn(0x7feaa00ad4b0 0x7feaa00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.597+0000 7feab57fa700 1 -- 172.21.15.26:0/1001516791 --> [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feaa8010ec0 con 0x7feaa00ad4b0 2024-04-21T10:29:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.597+0000 7feab57fa700 1 -- 172.21.15.26:0/1001516791 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7feaa80d0710 con 0x7feab8102a90 2024-04-21T10:29:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.597+0000 7feab77fe700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7feaa00a3cd0 0x7feaa00a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feaac006010 tx=0x7feaac00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.597+0000 7feabc810700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539] conn(0x7feaa00ad4b0 0x7feaa00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.597+0000 7feabc810700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539] conn(0x7feaa00ad4b0 0x7feaa00af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.598+0000 7feab57fa700 1 -- 172.21.15.26:0/1001516791 <== osd.2 v2:172.21.15.190:6800/2878992539 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feaa8010ec0 con 0x7feaa00ad4b0 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.615+0000 7feabe273700 1 -- 172.21.15.26:0/1001516791 --> [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feab8063650 con 0x7feaa00ad4b0 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.616+0000 7feab57fa700 1 -- 172.21.15.26:0/1001516791 <== osd.2 v2:172.21.15.190:6800/2878992539 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feab8063650 con 0x7feaa00ad4b0 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.616+0000 7feabe273700 1 -- 172.21.15.26:0/1001516791 >> [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539] conn(0x7feaa00ad4b0 msgr2=0x7feaa00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.616+0000 7feabe273700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539] conn(0x7feaa00ad4b0 0x7feaa00af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.616+0000 7feabe273700 1 -- 172.21.15.26:0/1001516791 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7feaa00a3cd0 msgr2=0x7feaa00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.616+0000 7feabe273700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7feaa00a3cd0 0x7feaa00a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feaac006010 tx=0x7feaac00b540 comp rx=0 tx=0).stop 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.616+0000 7feabe273700 1 -- 172.21.15.26:0/1001516791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab8102a90 msgr2=0x7feab81990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.616+0000 7feabe273700 1 --2- 172.21.15.26:0/1001516791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab8102a90 0x7feab81990f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feaa800b530 tx=0x7feaa800b8f0 comp rx=0 tx=0).stop 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.616+0000 7feabc810700 1 -- 172.21.15.26:0/1001516791 reap_dead start 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.616+0000 7feabe273700 1 -- 172.21.15.26:0/1001516791 shutdown_connections 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.616+0000 7feabe273700 1 -- 172.21.15.26:0/1001516791 >> 172.21.15.26:0/1001516791 conn(0x7feab8079ba0 msgr2=0x7feab8101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.616+0000 7feabe273700 1 -- 172.21.15.26:0/1001516791 shutdown_connections 2024-04-21T10:29:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:34.616+0000 7feabe273700 1 -- 172.21.15.26:0/1001516791 wait complete. 2024-04-21T10:29:35.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:35 smithi190 ceph-mon[31423]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-21T10:29:35.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:35 smithi026 ceph-mon[27965]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-21T10:29:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:35 smithi169 ceph-mon[31469]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-21T10:29:36.024 INFO:teuthology.orchestra.run.smithi026.stdout:77309411393 2024-04-21T10:29:36.024 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-21T10:29:36.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.619+0000 7f8d6dab5700 1 -- 172.21.15.26:0/179872310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d68102af0 msgr2=0x7f8d68102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:36.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.619+0000 7f8d6dab5700 1 --2- 172.21.15.26:0/179872310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d68102af0 0x7f8d68102f70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009990 tx=0x7f8d5c009ca0 comp rx=0 tx=0).stop 2024-04-21T10:29:36.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.619+0000 7f8d6dab5700 1 -- 172.21.15.26:0/179872310 shutdown_connections 2024-04-21T10:29:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.619+0000 7f8d6dab5700 1 --2- 172.21.15.26:0/179872310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d68108a80 0x7f8d68108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.619+0000 7f8d6dab5700 1 --2- 172.21.15.26:0/179872310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d681034b0 0x7f8d6810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.619+0000 7f8d6dab5700 1 --2- 172.21.15.26:0/179872310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d68102af0 0x7f8d68102f70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.619+0000 7f8d6dab5700 1 -- 172.21.15.26:0/179872310 >> 172.21.15.26:0/179872310 conn(0x7f8d68079b20 msgr2=0x7f8d68079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.619+0000 7f8d6dab5700 1 -- 172.21.15.26:0/179872310 shutdown_connections 2024-04-21T10:29:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.619+0000 7f8d6dab5700 1 -- 172.21.15.26:0/179872310 wait complete. 2024-04-21T10:29:36.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.620+0000 7f8d6dab5700 1 Processor -- start 2024-04-21T10:29:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.620+0000 7f8d6dab5700 1 -- start start 2024-04-21T10:29:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.620+0000 7f8d6dab5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d68102af0 0x7f8d68074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.620+0000 7f8d6dab5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d681034b0 0x7f8d68075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.620+0000 7f8d6dab5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d68108a80 0x7f8d680757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.620+0000 7f8d6dab5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8d68112d10 con 0x7f8d681034b0 2024-04-21T10:29:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.620+0000 7f8d6dab5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8d68112b90 con 0x7f8d68108a80 2024-04-21T10:29:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.620+0000 7f8d6dab5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8d68112e90 con 0x7f8d68102af0 2024-04-21T10:29:36.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.620+0000 7f8d66ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d681034b0 0x7f8d68075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.620+0000 7f8d66ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d681034b0 0x7f8d68075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54572/0 (socket says 172.21.15.26:54572) 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.620+0000 7f8d66ffd700 1 -- 172.21.15.26:0/1912849948 learned_addr learned my addr 172.21.15.26:0/1912849948 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d677fe700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d68102af0 0x7f8d68074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d67fff700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d68108a80 0x7f8d680757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d66ffd700 1 -- 172.21.15.26:0/1912849948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d68102af0 msgr2=0x7f8d68074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d66ffd700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d68102af0 0x7f8d68074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d66ffd700 1 -- 172.21.15.26:0/1912849948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d68108a80 msgr2=0x7f8d680757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d66ffd700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d68108a80 0x7f8d680757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d66ffd700 1 -- 172.21.15.26:0/1912849948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d681a7790 con 0x7f8d681034b0 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d66ffd700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d681034b0 0x7f8d68075280 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00b5c0 tx=0x7f8d5c004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d64ff9700 1 -- 172.21.15.26:0/1912849948 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c026070 con 0x7f8d681034b0 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d64ff9700 1 -- 172.21.15.26:0/1912849948 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d5c004170 con 0x7f8d681034b0 2024-04-21T10:29:36.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d6dab5700 1 -- 172.21.15.26:0/1912849948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d681a7a20 con 0x7f8d681034b0 2024-04-21T10:29:36.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d64ff9700 1 -- 172.21.15.26:0/1912849948 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c005250 con 0x7f8d681034b0 2024-04-21T10:29:36.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.621+0000 7f8d6dab5700 1 -- 172.21.15.26:0/1912849948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d681a7f00 con 0x7f8d681034b0 2024-04-21T10:29:36.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.622+0000 7f8d6dab5700 1 -- 172.21.15.26:0/1912849948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d680630d0 con 0x7f8d681034b0 2024-04-21T10:29:36.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.623+0000 7f8d64ff9700 1 -- 172.21.15.26:0/1912849948 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8d5c029070 con 0x7f8d681034b0 2024-04-21T10:29:36.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.624+0000 7f8d64ff9700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8d500a3cd0 0x7f8d500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:36.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.624+0000 7f8d677fe700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8d500a3cd0 0x7f8d500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:36.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.624+0000 7f8d64ff9700 1 -- 172.21.15.26:0/1912849948 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f8d5c0d13d0 con 0x7f8d681034b0 2024-04-21T10:29:36.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.624+0000 7f8d64ff9700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247] conn(0x7f8d500ad4b0 0x7f8d500af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:36.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.624+0000 7f8d64ff9700 1 -- 172.21.15.26:0/1912849948 --> [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d500aff80 con 0x7f8d500ad4b0 2024-04-21T10:29:36.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.624+0000 7f8d64ff9700 1 -- 172.21.15.26:0/1912849948 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f8d5c0d1780 con 0x7f8d681034b0 2024-04-21T10:29:36.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.624+0000 7f8d67fff700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247] conn(0x7f8d500ad4b0 0x7f8d500af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:36.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.624+0000 7f8d677fe700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8d500a3cd0 0x7f8d500a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d58006fd0 tx=0x7f8d58008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:36.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.625+0000 7f8d67fff700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247] conn(0x7f8d500ad4b0 0x7f8d500af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:36.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.625+0000 7f8d64ff9700 1 -- 172.21.15.26:0/1912849948 <== osd.9 v2:172.21.15.26:6826/3856298247 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d500aff80 con 0x7f8d500ad4b0 2024-04-21T10:29:36.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.642+0000 7f8d6dab5700 1 -- 172.21.15.26:0/1912849948 --> [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8d68063550 con 0x7f8d500ad4b0 2024-04-21T10:29:36.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.643+0000 7f8d64ff9700 1 -- 172.21.15.26:0/1912849948 <== osd.9 v2:172.21.15.26:6826/3856298247 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8d68063550 con 0x7f8d500ad4b0 2024-04-21T10:29:36.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.643+0000 7f8d6dab5700 1 -- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247] conn(0x7f8d500ad4b0 msgr2=0x7f8d500af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:36.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.643+0000 7f8d6dab5700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247] conn(0x7f8d500ad4b0 0x7f8d500af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:36.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.643+0000 7f8d6dab5700 1 -- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8d500a3cd0 msgr2=0x7f8d500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:36.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.643+0000 7f8d6dab5700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8d500a3cd0 0x7f8d500a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d58006fd0 tx=0x7f8d58008040 comp rx=0 tx=0).stop 2024-04-21T10:29:36.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.643+0000 7f8d6dab5700 1 -- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d681034b0 msgr2=0x7f8d68075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:36.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.643+0000 7f8d6dab5700 1 --2- 172.21.15.26:0/1912849948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d681034b0 0x7f8d68075280 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00b5c0 tx=0x7f8d5c004630 comp rx=0 tx=0).stop 2024-04-21T10:29:36.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.643+0000 7f8d67fff700 1 -- 172.21.15.26:0/1912849948 reap_dead start 2024-04-21T10:29:36.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.643+0000 7f8d6dab5700 1 -- 172.21.15.26:0/1912849948 shutdown_connections 2024-04-21T10:29:36.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.643+0000 7f8d6dab5700 1 -- 172.21.15.26:0/1912849948 >> 172.21.15.26:0/1912849948 conn(0x7f8d68079b20 msgr2=0x7f8d68101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:36.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.644+0000 7f8d6dab5700 1 -- 172.21.15.26:0/1912849948 shutdown_connections 2024-04-21T10:29:36.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:36.644+0000 7f8d6dab5700 1 -- 172.21.15.26:0/1912849948 wait complete. 2024-04-21T10:29:37.354 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:37 smithi026 ceph-mon[27965]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-21T10:29:37.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:37 smithi190 ceph-mon[31423]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-21T10:29:37.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:37 smithi169 ceph-mon[31469]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-21T10:29:38.189 INFO:teuthology.orchestra.run.smithi026.stdout:227633266727 2024-04-21T10:29:38.190 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-21T10:29:38.648 INFO:teuthology.orchestra.run.smithi026.stdout:317827579929 2024-04-21T10:29:38.649 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-21T10:29:38.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.321+0000 7f832ffee700 1 -- 172.21.15.26:0/3394518652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8328107960 msgr2=0x7f8328107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:38.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.321+0000 7f832ffee700 1 --2- 172.21.15.26:0/3394518652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8328107960 0x7f8328107d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8324009990 tx=0x7f8324009ca0 comp rx=0 tx=0).stop 2024-04-21T10:29:38.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.321+0000 7f832ffee700 1 -- 172.21.15.26:0/3394518652 shutdown_connections 2024-04-21T10:29:38.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.321+0000 7f832ffee700 1 --2- 172.21.15.26:0/3394518652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8328102320 0x7f832810e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:38.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.321+0000 7f832ffee700 1 --2- 172.21.15.26:0/3394518652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8328107960 0x7f8328107d40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.321+0000 7f832ffee700 1 --2- 172.21.15.26:0/3394518652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8328101960 0x7f8328101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.321+0000 7f832ffee700 1 -- 172.21.15.26:0/3394518652 >> 172.21.15.26:0/3394518652 conn(0x7f83280789e0 msgr2=0x7f8328078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.321+0000 7f832ffee700 1 -- 172.21.15.26:0/3394518652 shutdown_connections 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.321+0000 7f832ffee700 1 -- 172.21.15.26:0/3394518652 wait complete. 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832ffee700 1 Processor -- start 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832ffee700 1 -- start start 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832ffee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8328101960 0x7f8328197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832ffee700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8328102320 0x7f83281984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832ffee700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8328107960 0x7f832819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832ffee700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8328111b70 con 0x7f8328101960 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832ffee700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f83281119f0 con 0x7f8328107960 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832ffee700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8328111cf0 con 0x7f8328102320 2024-04-21T10:29:38.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832dd8a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8328101960 0x7f8328197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832dd8a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8328101960 0x7f8328197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54558/0 (socket says 172.21.15.26:54558) 2024-04-21T10:29:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832dd8a700 1 -- 172.21.15.26:0/1544230973 learned_addr learned my addr 172.21.15.26:0/1544230973 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832e58b700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8328107960 0x7f832819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832dd8a700 1 -- 172.21.15.26:0/1544230973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8328102320 msgr2=0x7f83281984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832dd8a700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8328102320 0x7f83281984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832dd8a700 1 -- 172.21.15.26:0/1544230973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8328107960 msgr2=0x7f832819c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832dd8a700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8328107960 0x7f832819c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832dd8a700 1 -- 172.21.15.26:0/1544230973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832819cf70 con 0x7f8328101960 2024-04-21T10:29:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832e58b700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8328107960 0x7f832819c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:29:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.322+0000 7f832dd8a700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8328101960 0x7f8328197f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8324000c00 tx=0x7f8324009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.323+0000 7f831effd700 1 -- 172.21.15.26:0/1544230973 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324027070 con 0x7f8328101960 2024-04-21T10:29:38.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.323+0000 7f832ffee700 1 -- 172.21.15.26:0/1544230973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832819d200 con 0x7f8328101960 2024-04-21T10:29:38.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.323+0000 7f831effd700 1 -- 172.21.15.26:0/1544230973 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8324022470 con 0x7f8328101960 2024-04-21T10:29:38.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.323+0000 7f831effd700 1 -- 172.21.15.26:0/1544230973 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324017650 con 0x7f8328101960 2024-04-21T10:29:38.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.323+0000 7f832ffee700 1 -- 172.21.15.26:0/1544230973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83281a4ae0 con 0x7f8328101960 2024-04-21T10:29:38.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.323+0000 7f832ffee700 1 -- 172.21.15.26:0/1544230973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8328104e10 con 0x7f8328101960 2024-04-21T10:29:38.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.325+0000 7f831effd700 1 -- 172.21.15.26:0/1544230973 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f832402a070 con 0x7f8328101960 2024-04-21T10:29:38.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.325+0000 7f831effd700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f83140a3cd0 0x7f83140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:38.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.326+0000 7f832d589700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f83140a3cd0 0x7f83140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:38.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.326+0000 7f831effd700 1 -- 172.21.15.26:0/1544230973 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f83240d1b50 con 0x7f8328101960 2024-04-21T10:29:38.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.326+0000 7f831effd700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313] conn(0x7f83140ad4b0 0x7f83140af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:38.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.326+0000 7f831effd700 1 -- 172.21.15.26:0/1544230973 --> [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f832400ce90 con 0x7f83140ad4b0 2024-04-21T10:29:38.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.326+0000 7f831effd700 1 -- 172.21.15.26:0/1544230973 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f83240d1f00 con 0x7f8328101960 2024-04-21T10:29:38.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.326+0000 7f832d589700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f83140a3cd0 0x7f83140a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8318005fd0 tx=0x7f8318005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:38.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.326+0000 7f832e58b700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313] conn(0x7f83140ad4b0 0x7f83140af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:38.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.327+0000 7f832e58b700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313] conn(0x7f83140ad4b0 0x7f83140af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:38.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.328+0000 7f831effd700 1 -- 172.21.15.26:0/1544230973 <== osd.1 v2:172.21.15.169:6800/4294740313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f832400ce90 con 0x7f83140ad4b0 2024-04-21T10:29:38.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.344+0000 7f832ffee700 1 -- 172.21.15.26:0/1544230973 --> [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83281030a0 con 0x7f83140ad4b0 2024-04-21T10:29:38.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.344+0000 7f831effd700 1 -- 172.21.15.26:0/1544230973 <== osd.1 v2:172.21.15.169:6800/4294740313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83281030a0 con 0x7f83140ad4b0 2024-04-21T10:29:38.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.345+0000 7f832ffee700 1 -- 172.21.15.26:0/1544230973 >> [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313] conn(0x7f83140ad4b0 msgr2=0x7f83140af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:38.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.345+0000 7f832ffee700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313] conn(0x7f83140ad4b0 0x7f83140af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:38.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.345+0000 7f832ffee700 1 -- 172.21.15.26:0/1544230973 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f83140a3cd0 msgr2=0x7f83140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.345+0000 7f832ffee700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f83140a3cd0 0x7f83140a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8318005fd0 tx=0x7f8318005d00 comp rx=0 tx=0).stop 2024-04-21T10:29:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.345+0000 7f832ffee700 1 -- 172.21.15.26:0/1544230973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8328101960 msgr2=0x7f8328197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.345+0000 7f832ffee700 1 --2- 172.21.15.26:0/1544230973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8328101960 0x7f8328197f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8324000c00 tx=0x7f8324009580 comp rx=0 tx=0).stop 2024-04-21T10:29:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.345+0000 7f832e58b700 1 -- 172.21.15.26:0/1544230973 reap_dead start 2024-04-21T10:29:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.345+0000 7f832ffee700 1 -- 172.21.15.26:0/1544230973 shutdown_connections 2024-04-21T10:29:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.345+0000 7f832ffee700 1 -- 172.21.15.26:0/1544230973 >> 172.21.15.26:0/1544230973 conn(0x7f83280789e0 msgr2=0x7f83281043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.345+0000 7f832ffee700 1 -- 172.21.15.26:0/1544230973 shutdown_connections 2024-04-21T10:29:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:35.345+0000 7f832ffee700 1 -- 172.21.15.26:0/1544230973 wait complete. 2024-04-21T10:29:39.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:39 smithi026 ceph-mon[27965]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-21T10:29:39.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:39 smithi190 ceph-mon[31423]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-21T10:29:39.589 INFO:teuthology.orchestra.run.smithi026.stdout:55834574919 2024-04-21T10:29:39.590 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-21T10:29:39.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:39 smithi169 ceph-mon[31469]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-21T10:29:39.964 INFO:teuthology.orchestra.run.smithi026.stdout:146028888120 2024-04-21T10:29:39.965 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-21T10:29:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.278+0000 7ff43daf6700 1 -- 172.21.15.26:0/2019609329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4381013e0 msgr2=0x7ff438101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.278+0000 7ff43daf6700 1 --2- 172.21.15.26:0/2019609329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4381013e0 0x7ff438101860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009990 tx=0x7ff42c009ca0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.278+0000 7ff43daf6700 1 -- 172.21.15.26:0/2019609329 shutdown_connections 2024-04-21T10:29:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.278+0000 7ff43daf6700 1 --2- 172.21.15.26:0/2019609329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff438101da0 0x7ff43810f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.278+0000 7ff43daf6700 1 --2- 172.21.15.26:0/2019609329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff438107b00 0x7ff438107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.278+0000 7ff43daf6700 1 --2- 172.21.15.26:0/2019609329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4381013e0 0x7ff438101860 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.278+0000 7ff43daf6700 1 -- 172.21.15.26:0/2019609329 >> 172.21.15.26:0/2019609329 conn(0x7ff438076c60 msgr2=0x7ff438077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.278+0000 7ff43daf6700 1 -- 172.21.15.26:0/2019609329 shutdown_connections 2024-04-21T10:29:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.278+0000 7ff43daf6700 1 -- 172.21.15.26:0/2019609329 wait complete. 2024-04-21T10:29:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.278+0000 7ff43daf6700 1 Processor -- start 2024-04-21T10:29:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff43daf6700 1 -- start start 2024-04-21T10:29:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff43daf6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4381013e0 0x7ff4381991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff43daf6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff438101da0 0x7ff438199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff43daf6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff438107b00 0x7ff43819da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff43daf6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff438112c90 con 0x7ff438107b00 2024-04-21T10:29:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff43daf6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff438112b10 con 0x7ff4381013e0 2024-04-21T10:29:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff43daf6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff438112e10 con 0x7ff438101da0 2024-04-21T10:29:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff437fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff438107b00 0x7ff43819da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff437fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff438107b00 0x7ff43819da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54594/0 (socket says 172.21.15.26:54594) 2024-04-21T10:29:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff437fff700 1 -- 172.21.15.26:0/2654137797 learned_addr learned my addr 172.21.15.26:0/2654137797 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff436ffd700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff438101da0 0x7ff438199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff4377fe700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4381013e0 0x7ff4381991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff437fff700 1 -- 172.21.15.26:0/2654137797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff438101da0 msgr2=0x7ff438199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff437fff700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff438101da0 0x7ff438199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff437fff700 1 -- 172.21.15.26:0/2654137797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4381013e0 msgr2=0x7ff4381991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff437fff700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4381013e0 0x7ff4381991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff437fff700 1 -- 172.21.15.26:0/2654137797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43819e1d0 con 0x7ff438107b00 2024-04-21T10:29:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.279+0000 7ff437fff700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff438107b00 0x7ff43819da90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff43000ada0 tx=0x7ff43000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.280+0000 7ff434ff9700 1 -- 172.21.15.26:0/2654137797 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430017020 con 0x7ff438107b00 2024-04-21T10:29:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.280+0000 7ff434ff9700 1 -- 172.21.15.26:0/2654137797 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff43000cd80 con 0x7ff438107b00 2024-04-21T10:29:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.280+0000 7ff43daf6700 1 -- 172.21.15.26:0/2654137797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43819e4c0 con 0x7ff438107b00 2024-04-21T10:29:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.280+0000 7ff43daf6700 1 -- 172.21.15.26:0/2654137797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4381a5d40 con 0x7ff438107b00 2024-04-21T10:29:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.280+0000 7ff434ff9700 1 -- 172.21.15.26:0/2654137797 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430005260 con 0x7ff438107b00 2024-04-21T10:29:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.280+0000 7ff43daf6700 1 -- 172.21.15.26:0/2654137797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff424001030 con 0x7ff438107b00 2024-04-21T10:29:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.282+0000 7ff434ff9700 1 -- 172.21.15.26:0/2654137797 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff43000c8d0 con 0x7ff438107b00 2024-04-21T10:29:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.283+0000 7ff434ff9700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff4200a3cd0 0x7ff4200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.283+0000 7ff4377fe700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff4200a3cd0 0x7ff4200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.283+0000 7ff434ff9700 1 -- 172.21.15.26:0/2654137797 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7ff4300d06e0 con 0x7ff438107b00 2024-04-21T10:29:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.283+0000 7ff434ff9700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337] conn(0x7ff4200ad4b0 0x7ff4200af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.283+0000 7ff434ff9700 1 -- 172.21.15.26:0/2654137797 --> [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff430018e70 con 0x7ff4200ad4b0 2024-04-21T10:29:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.283+0000 7ff434ff9700 1 -- 172.21.15.26:0/2654137797 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7ff4300d0bc0 con 0x7ff438107b00 2024-04-21T10:29:40.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.283+0000 7ff436ffd700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337] conn(0x7ff4200ad4b0 0x7ff4200af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:40.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.284+0000 7ff4377fe700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff4200a3cd0 0x7ff4200a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff4380781e0 tx=0x7ff42800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:40.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.284+0000 7ff436ffd700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337] conn(0x7ff4200ad4b0 0x7ff4200af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:40.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.284+0000 7ff434ff9700 1 -- 172.21.15.26:0/2654137797 <== osd.6 v2:172.21.15.26:6818/4211294337 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff430018e70 con 0x7ff4200ad4b0 2024-04-21T10:29:40.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.301+0000 7ff43daf6700 1 -- 172.21.15.26:0/2654137797 --> [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff424002e60 con 0x7ff4200ad4b0 2024-04-21T10:29:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.301+0000 7ff434ff9700 1 -- 172.21.15.26:0/2654137797 <== osd.6 v2:172.21.15.26:6818/4211294337 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff424002e60 con 0x7ff4200ad4b0 2024-04-21T10:29:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.301+0000 7ff43daf6700 1 -- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337] conn(0x7ff4200ad4b0 msgr2=0x7ff4200af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.301+0000 7ff43daf6700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337] conn(0x7ff4200ad4b0 0x7ff4200af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.301+0000 7ff43daf6700 1 -- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff4200a3cd0 msgr2=0x7ff4200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.301+0000 7ff43daf6700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff4200a3cd0 0x7ff4200a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff4380781e0 tx=0x7ff42800b410 comp rx=0 tx=0).stop 2024-04-21T10:29:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.301+0000 7ff43daf6700 1 -- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff438107b00 msgr2=0x7ff43819da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.301+0000 7ff43daf6700 1 --2- 172.21.15.26:0/2654137797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff438107b00 0x7ff43819da90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff43000ada0 tx=0x7ff43000c330 comp rx=0 tx=0).stop 2024-04-21T10:29:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.302+0000 7ff437fff700 1 -- 172.21.15.26:0/2654137797 reap_dead start 2024-04-21T10:29:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.302+0000 7ff43daf6700 1 -- 172.21.15.26:0/2654137797 shutdown_connections 2024-04-21T10:29:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.302+0000 7ff43daf6700 1 -- 172.21.15.26:0/2654137797 >> 172.21.15.26:0/2654137797 conn(0x7ff438076c60 msgr2=0x7ff438100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.302+0000 7ff43daf6700 1 -- 172.21.15.26:0/2654137797 shutdown_connections 2024-04-21T10:29:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.302+0000 7ff43daf6700 1 -- 172.21.15.26:0/2654137797 wait complete. 2024-04-21T10:29:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.923+0000 7ff88f59e700 1 -- 172.21.15.26:0/955163061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff888102230 msgr2=0x7ff88810e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.923+0000 7ff88f59e700 1 --2- 172.21.15.26:0/955163061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff888102230 0x7ff88810e740 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff88400b0a0 tx=0x7ff88400b3b0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.923+0000 7ff88f59e700 1 -- 172.21.15.26:0/955163061 shutdown_connections 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.923+0000 7ff88f59e700 1 --2- 172.21.15.26:0/955163061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff888107870 0x7ff888107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.923+0000 7ff88f59e700 1 --2- 172.21.15.26:0/955163061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff888102230 0x7ff88810e740 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.923+0000 7ff88f59e700 1 --2- 172.21.15.26:0/955163061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff888101870 0x7ff888101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.923+0000 7ff88f59e700 1 -- 172.21.15.26:0/955163061 >> 172.21.15.26:0/955163061 conn(0x7ff8880fd360 msgr2=0x7ff8880ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.923+0000 7ff88f59e700 1 -- 172.21.15.26:0/955163061 shutdown_connections 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.923+0000 7ff88f59e700 1 -- 172.21.15.26:0/955163061 wait complete. 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88f59e700 1 Processor -- start 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88f59e700 1 -- start start 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88f59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff888101870 0x7ff888198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff888102230 0x7ff888199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88f59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff888107870 0x7ff88819d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88f59e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff888111ab0 con 0x7ff888101870 2024-04-21T10:29:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff888111930 con 0x7ff888102230 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88f59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff888111c30 con 0x7ff888107870 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff888101870 0x7ff888198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff888101870 0x7ff888198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54620/0 (socket says 172.21.15.26:54620) 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88e59c700 1 -- 172.21.15.26:0/2408659209 learned_addr learned my addr 172.21.15.26:0/2408659209 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88ed9d700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff888107870 0x7ff88819d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88dd9b700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff888102230 0x7ff888199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.924+0000 7ff88e59c700 1 -- 172.21.15.26:0/2408659209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff888107870 msgr2=0x7ff88819d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.925+0000 7ff88e59c700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff888107870 0x7ff88819d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.925+0000 7ff88e59c700 1 -- 172.21.15.26:0/2408659209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff888102230 msgr2=0x7ff888199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.925+0000 7ff88e59c700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff888102230 0x7ff888199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.925+0000 7ff88e59c700 1 -- 172.21.15.26:0/2408659209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88819dfd0 con 0x7ff888101870 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.925+0000 7ff88e59c700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff888101870 0x7ff888198fc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e9a0 tx=0x7ff87c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.925+0000 7ff87b7fe700 1 -- 172.21.15.26:0/2408659209 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00ca10 con 0x7ff888101870 2024-04-21T10:29:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.925+0000 7ff87b7fe700 1 -- 172.21.15.26:0/2408659209 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff87c00cbb0 con 0x7ff888101870 2024-04-21T10:29:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.925+0000 7ff87b7fe700 1 -- 172.21.15.26:0/2408659209 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c018680 con 0x7ff888101870 2024-04-21T10:29:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.925+0000 7ff88f59e700 1 -- 172.21.15.26:0/2408659209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88819e2c0 con 0x7ff888101870 2024-04-21T10:29:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.925+0000 7ff88f59e700 1 -- 172.21.15.26:0/2408659209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8881a5b40 con 0x7ff888101870 2024-04-21T10:29:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.926+0000 7ff88f59e700 1 -- 172.21.15.26:0/2408659209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff888063150 con 0x7ff888101870 2024-04-21T10:29:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.927+0000 7ff87b7fe700 1 -- 172.21.15.26:0/2408659209 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff87c010430 con 0x7ff888101870 2024-04-21T10:29:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.928+0000 7ff87b7fe700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff8740a3d50 0x7ff8740a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.928+0000 7ff88dd9b700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff8740a3d50 0x7ff8740a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.928+0000 7ff87b7fe700 1 -- 172.21.15.26:0/2408659209 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7ff87c0d09c0 con 0x7ff888101870 2024-04-21T10:29:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.928+0000 7ff87b7fe700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169] conn(0x7ff8740ad530 0x7ff8740af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.928+0000 7ff87b7fe700 1 -- 172.21.15.26:0/2408659209 --> [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff87c019ec0 con 0x7ff8740ad530 2024-04-21T10:29:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.928+0000 7ff87b7fe700 1 -- 172.21.15.26:0/2408659209 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7ff87c0d0ea0 con 0x7ff888101870 2024-04-21T10:29:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.929+0000 7ff88ed9d700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169] conn(0x7ff8740ad530 0x7ff8740af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.929+0000 7ff88dd9b700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff8740a3d50 0x7ff8740a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff880009680 tx=0x7ff880009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.929+0000 7ff88ed9d700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169] conn(0x7ff8740ad530 0x7ff8740af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.930+0000 7ff87b7fe700 1 -- 172.21.15.26:0/2408659209 <== osd.3 v2:172.21.15.26:6810/2335886169 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff87c019ec0 con 0x7ff8740ad530 2024-04-21T10:29:40.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff88f59e700 1 -- 172.21.15.26:0/2408659209 --> [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff888063550 con 0x7ff8740ad530 2024-04-21T10:29:40.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff87b7fe700 1 -- 172.21.15.26:0/2408659209 <== osd.3 v2:172.21.15.26:6810/2335886169 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff888063550 con 0x7ff8740ad530 2024-04-21T10:29:40.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff88f59e700 1 -- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169] conn(0x7ff8740ad530 msgr2=0x7ff8740af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff88f59e700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169] conn(0x7ff8740ad530 0x7ff8740af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff88f59e700 1 -- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff8740a3d50 msgr2=0x7ff8740a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff88f59e700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7ff8740a3d50 0x7ff8740a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff880009680 tx=0x7ff880009260 comp rx=0 tx=0).stop 2024-04-21T10:29:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff88f59e700 1 -- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff888101870 msgr2=0x7ff888198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff88f59e700 1 --2- 172.21.15.26:0/2408659209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff888101870 0x7ff888198fc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e9a0 tx=0x7ff87c00ecb0 comp rx=0 tx=0).stop 2024-04-21T10:29:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff88ed9d700 1 -- 172.21.15.26:0/2408659209 reap_dead start 2024-04-21T10:29:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff88f59e700 1 -- 172.21.15.26:0/2408659209 shutdown_connections 2024-04-21T10:29:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff88f59e700 1 -- 172.21.15.26:0/2408659209 >> 172.21.15.26:0/2408659209 conn(0x7ff8880fd360 msgr2=0x7ff88806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff88f59e700 1 -- 172.21.15.26:0/2408659209 shutdown_connections 2024-04-21T10:29:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:40.947+0000 7ff88f59e700 1 -- 172.21.15.26:0/2408659209 wait complete. 2024-04-21T10:29:41.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:41 smithi190 ceph-mon[31423]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:29:41.621 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:41 smithi026 ceph-mon[27965]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:29:41.651 INFO:teuthology.orchestra.run.smithi026.stdout:111669149757 2024-04-21T10:29:41.651 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-21T10:29:41.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:41 smithi169 ceph-mon[31469]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:29:42.844 INFO:teuthology.orchestra.run.smithi026.stdout:296352743455 2024-04-21T10:29:42.844 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-21T10:29:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.417+0000 7f48b1f39700 1 -- 172.21.15.26:0/1255901468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48ac06a750 msgr2=0x7f48ac06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.417+0000 7f48b1f39700 1 --2- 172.21.15.26:0/1255901468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48ac06a750 0x7f48ac06abb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009990 tx=0x7f48a0009ca0 comp rx=0 tx=0).stop 2024-04-21T10:29:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.417+0000 7f48b1f39700 1 -- 172.21.15.26:0/1255901468 shutdown_connections 2024-04-21T10:29:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.417+0000 7f48b1f39700 1 --2- 172.21.15.26:0/1255901468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48ac108a30 0x7f48ac108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.417+0000 7f48b1f39700 1 --2- 172.21.15.26:0/1255901468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48ac06b0f0 0x7f48ac10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.417+0000 7f48b1f39700 1 --2- 172.21.15.26:0/1255901468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48ac06a750 0x7f48ac06abb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.417+0000 7f48b1f39700 1 -- 172.21.15.26:0/1255901468 >> 172.21.15.26:0/1255901468 conn(0x7f48ac0788f0 msgr2=0x7f48ac078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.417+0000 7f48b1f39700 1 -- 172.21.15.26:0/1255901468 shutdown_connections 2024-04-21T10:29:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.417+0000 7f48b1f39700 1 -- 172.21.15.26:0/1255901468 wait complete. 2024-04-21T10:29:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.418+0000 7f48b1f39700 1 Processor -- start 2024-04-21T10:29:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.418+0000 7f48b1f39700 1 -- start start 2024-04-21T10:29:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.418+0000 7f48b1f39700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48ac06a750 0x7f48ac101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.418+0000 7f48b1f39700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48ac06b0f0 0x7f48ac101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.418+0000 7f48b1f39700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48ac108a30 0x7f48ac102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.418+0000 7f48b1f39700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac112cb0 con 0x7f48ac108a30 2024-04-21T10:29:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.418+0000 7f48b1f39700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac112b30 con 0x7f48ac06b0f0 2024-04-21T10:29:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.418+0000 7f48b1f39700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac112e30 con 0x7f48ac06a750 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.418+0000 7f48abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48ac108a30 0x7f48ac102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.418+0000 7f48abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48ac108a30 0x7f48ac102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54646/0 (socket says 172.21.15.26:54646) 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.418+0000 7f48abfff700 1 -- 172.21.15.26:0/3998925015 learned_addr learned my addr 172.21.15.26:0/3998925015 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.418+0000 7f48ab7fe700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48ac06a750 0x7f48ac101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48aaffd700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48ac06b0f0 0x7f48ac101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48abfff700 1 -- 172.21.15.26:0/3998925015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48ac06a750 msgr2=0x7f48ac101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48abfff700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48ac06a750 0x7f48ac101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48abfff700 1 -- 172.21.15.26:0/3998925015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48ac06b0f0 msgr2=0x7f48ac101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48abfff700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48ac06b0f0 0x7f48ac101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48abfff700 1 -- 172.21.15.26:0/3998925015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac1a77d0 con 0x7f48ac108a30 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48aaffd700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48ac06b0f0 0x7f48ac101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48abfff700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48ac108a30 0x7f48ac102490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f48a400b970 tx=0x7f48a400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48a8ff9700 1 -- 172.21.15.26:0/3998925015 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4019070 con 0x7f48ac108a30 2024-04-21T10:29:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48b1f39700 1 -- 172.21.15.26:0/3998925015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ac1a7ac0 con 0x7f48ac108a30 2024-04-21T10:29:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48a8ff9700 1 -- 172.21.15.26:0/3998925015 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48a400cde0 con 0x7f48ac108a30 2024-04-21T10:29:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48b1f39700 1 -- 172.21.15.26:0/3998925015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ac1a8050 con 0x7f48ac108a30 2024-04-21T10:29:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.419+0000 7f48a8ff9700 1 -- 172.21.15.26:0/3998925015 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a4007370 con 0x7f48ac108a30 2024-04-21T10:29:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.420+0000 7f48b1f39700 1 -- 172.21.15.26:0/3998925015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48ac0630d0 con 0x7f48ac108a30 2024-04-21T10:29:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.421+0000 7f48a8ff9700 1 -- 172.21.15.26:0/3998925015 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f48a401c070 con 0x7f48ac108a30 2024-04-21T10:29:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.422+0000 7f48a8ff9700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f48940a3cd0 0x7f48940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.422+0000 7f48ab7fe700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f48940a3cd0 0x7f48940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.422+0000 7f48a8ff9700 1 -- 172.21.15.26:0/3998925015 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f48a40d0180 con 0x7f48ac108a30 2024-04-21T10:29:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.422+0000 7f48a8ff9700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578] conn(0x7f48940ad4b0 0x7f48940af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.422+0000 7f48a8ff9700 1 -- 172.21.15.26:0/3998925015 --> [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48940aff80 con 0x7f48940ad4b0 2024-04-21T10:29:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.422+0000 7f48a8ff9700 1 -- 172.21.15.26:0/3998925015 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f48a40d0660 con 0x7f48ac108a30 2024-04-21T10:29:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.422+0000 7f48ab7fe700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f48940a3cd0 0x7f48940a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f489c009a10 tx=0x7f489c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.423+0000 7f48aaffd700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578] conn(0x7f48940ad4b0 0x7f48940af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.423+0000 7f48aaffd700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578] conn(0x7f48940ad4b0 0x7f48940af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.424+0000 7f48a8ff9700 1 -- 172.21.15.26:0/3998925015 <== osd.8 v2:172.21.15.190:6816/250015578 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48940aff80 con 0x7f48940ad4b0 2024-04-21T10:29:43.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.441+0000 7f48b1f39700 1 -- 172.21.15.26:0/3998925015 --> [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f48ac0634d0 con 0x7f48940ad4b0 2024-04-21T10:29:43.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.441+0000 7f48a8ff9700 1 -- 172.21.15.26:0/3998925015 <== osd.8 v2:172.21.15.190:6816/250015578 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f48ac0634d0 con 0x7f48940ad4b0 2024-04-21T10:29:43.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.441+0000 7f48b1f39700 1 -- 172.21.15.26:0/3998925015 >> [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578] conn(0x7f48940ad4b0 msgr2=0x7f48940af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:43.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.441+0000 7f48b1f39700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578] conn(0x7f48940ad4b0 0x7f48940af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:43.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.441+0000 7f48b1f39700 1 -- 172.21.15.26:0/3998925015 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f48940a3cd0 msgr2=0x7f48940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:43.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.441+0000 7f48b1f39700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f48940a3cd0 0x7f48940a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f489c009a10 tx=0x7f489c009190 comp rx=0 tx=0).stop 2024-04-21T10:29:43.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.442+0000 7f48b1f39700 1 -- 172.21.15.26:0/3998925015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48ac108a30 msgr2=0x7f48ac102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:43.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.442+0000 7f48b1f39700 1 --2- 172.21.15.26:0/3998925015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48ac108a30 0x7f48ac102490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f48a400b970 tx=0x7f48a400bc80 comp rx=0 tx=0).stop 2024-04-21T10:29:43.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.442+0000 7f48abfff700 1 -- 172.21.15.26:0/3998925015 reap_dead start 2024-04-21T10:29:43.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.442+0000 7f48b1f39700 1 -- 172.21.15.26:0/3998925015 shutdown_connections 2024-04-21T10:29:43.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.442+0000 7f48b1f39700 1 -- 172.21.15.26:0/3998925015 >> 172.21.15.26:0/3998925015 conn(0x7f48ac0788f0 msgr2=0x7f48ac0ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:43.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.442+0000 7f48b1f39700 1 -- 172.21.15.26:0/3998925015 shutdown_connections 2024-04-21T10:29:43.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:43.442+0000 7f48b1f39700 1 -- 172.21.15.26:0/3998925015 wait complete. 2024-04-21T10:29:43.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:43 smithi026 ceph-mon[27965]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:29:43.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:43 smithi190 ceph-mon[31423]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:29:43.546 INFO:teuthology.orchestra.run.smithi026.stdout:206158430254 2024-04-21T10:29:43.546 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-21T10:29:43.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:43 smithi169 ceph-mon[31469]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:29:44.143 INFO:teuthology.orchestra.run.smithi026.stdout:261993005093 2024-04-21T10:29:44.143 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-21T10:29:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.145+0000 7f3be759e700 1 -- 172.21.15.26:0/1263414254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be8108a20 msgr2=0x7f3be8108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.145+0000 7f3be759e700 1 --2- 172.21.15.26:0/1263414254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be8108a20 0x7f3be8108e00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009990 tx=0x7f3bd8009ca0 comp rx=0 tx=0).stop 2024-04-21T10:29:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.145+0000 7f3be759e700 1 -- 172.21.15.26:0/1263414254 shutdown_connections 2024-04-21T10:29:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.145+0000 7f3be759e700 1 --2- 172.21.15.26:0/1263414254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be806b970 0x7f3be810f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.145+0000 7f3be759e700 1 --2- 172.21.15.26:0/1263414254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be8108a20 0x7f3be8108e00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.145+0000 7f3be759e700 1 --2- 172.21.15.26:0/1263414254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3be806afb0 0x7f3be806b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:45.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.145+0000 7f3be759e700 1 -- 172.21.15.26:0/1263414254 >> 172.21.15.26:0/1263414254 conn(0x7f3be8078780 msgr2=0x7f3be8078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:45.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.145+0000 7f3be759e700 1 -- 172.21.15.26:0/1263414254 shutdown_connections 2024-04-21T10:29:45.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.145+0000 7f3be759e700 1 -- 172.21.15.26:0/1263414254 wait complete. 2024-04-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be759e700 1 Processor -- start 2024-04-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be759e700 1 -- start start 2024-04-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be759e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be806afb0 0x7f3be8101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3be806b970 0x7f3be8102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be759e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be8108a20 0x7f3be8102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be759e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3be8112b70 con 0x7f3be8108a20 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81129f0 con 0x7f3be806b970 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be759e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3be8112cf0 con 0x7f3be806afb0 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be8108a20 0x7f3be8102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be8108a20 0x7f3be8102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50648/0 (socket says 172.21.15.26:50648) 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be6d9d700 1 -- 172.21.15.26:0/3184856321 learned_addr learned my addr 172.21.15.26:0/3184856321 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be659c700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be806afb0 0x7f3be8101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be6d9d700 1 -- 172.21.15.26:0/3184856321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be806afb0 msgr2=0x7f3be8101bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be6d9d700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be806afb0 0x7f3be8101bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be6d9d700 1 -- 172.21.15.26:0/3184856321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3be806b970 msgr2=0x7f3be8102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be6d9d700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3be806b970 0x7f3be8102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be6d9d700 1 -- 172.21.15.26:0/3184856321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be81a7580 con 0x7f3be8108a20 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.146+0000 7f3be6d9d700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be8108a20 0x7f3be8102650 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3be0007f10 tx=0x7f3be000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:45.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.147+0000 7f3bd77fe700 1 -- 172.21.15.26:0/3184856321 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0009e20 con 0x7f3be8108a20 2024-04-21T10:29:45.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.147+0000 7f3be759e700 1 -- 172.21.15.26:0/3184856321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be81a7870 con 0x7f3be8108a20 2024-04-21T10:29:45.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.147+0000 7f3bd77fe700 1 -- 172.21.15.26:0/3184856321 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3be000dbd0 con 0x7f3be8108a20 2024-04-21T10:29:45.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.147+0000 7f3bd77fe700 1 -- 172.21.15.26:0/3184856321 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0005990 con 0x7f3be8108a20 2024-04-21T10:29:45.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.147+0000 7f3be759e700 1 -- 172.21.15.26:0/3184856321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be81a7db0 con 0x7f3be8108a20 2024-04-21T10:29:45.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.148+0000 7f3be759e700 1 -- 172.21.15.26:0/3184856321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3be80630d0 con 0x7f3be8108a20 2024-04-21T10:29:45.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.149+0000 7f3bd77fe700 1 -- 172.21.15.26:0/3184856321 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3be001a020 con 0x7f3be8108a20 2024-04-21T10:29:45.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.149+0000 7f3bd77fe700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3bd00a3cd0 0x7f3bd00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:45.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.150+0000 7f3be659c700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3bd00a3cd0 0x7f3bd00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:45.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.150+0000 7f3bd77fe700 1 -- 172.21.15.26:0/3184856321 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f3be00d0140 con 0x7f3be8108a20 2024-04-21T10:29:45.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.150+0000 7f3bd77fe700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638] conn(0x7f3bd00ad4b0 0x7f3bd00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:45.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.150+0000 7f3bd77fe700 1 -- 172.21.15.26:0/3184856321 --> [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3bd00aff80 con 0x7f3bd00ad4b0 2024-04-21T10:29:45.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.150+0000 7f3bd77fe700 1 -- 172.21.15.26:0/3184856321 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f3be00d0620 con 0x7f3be8108a20 2024-04-21T10:29:45.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.150+0000 7f3be659c700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3bd00a3cd0 0x7f3bd00a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800b5c0 tx=0x7f3bd8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:45.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.150+0000 7f3be5d9b700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638] conn(0x7f3bd00ad4b0 0x7f3bd00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:45.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.151+0000 7f3be5d9b700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638] conn(0x7f3bd00ad4b0 0x7f3bd00af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:45.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.152+0000 7f3bd77fe700 1 -- 172.21.15.26:0/3184856321 <== osd.11 v2:172.21.15.190:6824/1221661638 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bd00aff80 con 0x7f3bd00ad4b0 2024-04-21T10:29:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.168+0000 7f3be759e700 1 -- 172.21.15.26:0/3184856321 --> [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3be8063550 con 0x7f3bd00ad4b0 2024-04-21T10:29:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.169+0000 7f3bd77fe700 1 -- 172.21.15.26:0/3184856321 <== osd.11 v2:172.21.15.190:6824/1221661638 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3be8063550 con 0x7f3bd00ad4b0 2024-04-21T10:29:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.169+0000 7f3be759e700 1 -- 172.21.15.26:0/3184856321 >> [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638] conn(0x7f3bd00ad4b0 msgr2=0x7f3bd00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.169+0000 7f3be759e700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638] conn(0x7f3bd00ad4b0 0x7f3bd00af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.169+0000 7f3be759e700 1 -- 172.21.15.26:0/3184856321 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3bd00a3cd0 msgr2=0x7f3bd00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.169+0000 7f3be759e700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3bd00a3cd0 0x7f3bd00a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800b5c0 tx=0x7f3bd8005960 comp rx=0 tx=0).stop 2024-04-21T10:29:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.169+0000 7f3be759e700 1 -- 172.21.15.26:0/3184856321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be8108a20 msgr2=0x7f3be8102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.169+0000 7f3be759e700 1 --2- 172.21.15.26:0/3184856321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3be8108a20 0x7f3be8102650 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3be0007f10 tx=0x7f3be000d3b0 comp rx=0 tx=0).stop 2024-04-21T10:29:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.169+0000 7f3be6d9d700 1 -- 172.21.15.26:0/3184856321 reap_dead start 2024-04-21T10:29:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.169+0000 7f3be759e700 1 -- 172.21.15.26:0/3184856321 shutdown_connections 2024-04-21T10:29:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.169+0000 7f3be759e700 1 -- 172.21.15.26:0/3184856321 >> 172.21.15.26:0/3184856321 conn(0x7f3be8078780 msgr2=0x7f3be80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.169+0000 7f3be759e700 1 -- 172.21.15.26:0/3184856321 shutdown_connections 2024-04-21T10:29:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:45.169+0000 7f3be759e700 1 -- 172.21.15.26:0/3184856321 wait complete. 2024-04-21T10:29:45.308 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:45 smithi026 ceph-mon[27965]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:29:45.510 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:45 smithi169 ceph-mon[31469]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:29:45.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:45 smithi190 ceph-mon[31423]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:29:45.911 INFO:teuthology.orchestra.run.smithi026.stdout:352187318295 2024-04-21T10:29:45.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-21T10:29:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:46.166 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:46.168 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:46.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.180+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2621672345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce006ac90 msgr2=0x7fbce006b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:46.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.180+0000 7fbce6b4a700 1 --2- 172.21.15.26:0/2621672345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce006ac90 0x7fbce006b0f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbcd001a9a0 tx=0x7fbcd001acb0 comp rx=0 tx=0).stop 2024-04-21T10:29:46.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.183+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2621672345 shutdown_connections 2024-04-21T10:29:46.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.183+0000 7fbce6b4a700 1 --2- 172.21.15.26:0/2621672345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbce0107970 0x7fbce006a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:46.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.183+0000 7fbce6b4a700 1 --2- 172.21.15.26:0/2621672345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbce0105480 0x7fbce0105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:46.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.183+0000 7fbce6b4a700 1 --2- 172.21.15.26:0/2621672345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce006ac90 0x7fbce006b0f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:46.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.183+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2621672345 >> 172.21.15.26:0/2621672345 conn(0x7fbce00788f0 msgr2=0x7fbce0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:46.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.186+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2621672345 shutdown_connections 2024-04-21T10:29:46.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.186+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2621672345 wait complete. 2024-04-21T10:29:46.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.187+0000 7fbce6b4a700 1 Processor -- start 2024-04-21T10:29:46.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.187+0000 7fbce6b4a700 1 -- start start 2024-04-21T10:29:46.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.188+0000 7fbce6b4a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbce006ac90 0x7fbce0073430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:46.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.188+0000 7fbce6b4a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbce0105480 0x7fbce0073970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:46.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.188+0000 7fbce6b4a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce0107970 0x7fbce0116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:46.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.188+0000 7fbce6b4a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbce010a780 con 0x7fbce0107970 2024-04-21T10:29:46.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.188+0000 7fbce6b4a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbce010a600 con 0x7fbce0105480 2024-04-21T10:29:46.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.188+0000 7fbce6b4a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbce010a900 con 0x7fbce006ac90 2024-04-21T10:29:46.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.188+0000 7fbcdffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbce0105480 0x7fbce0073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:46.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.189+0000 7fbcdffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbce0105480 0x7fbce0073970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.26:47588/0 (socket says 172.21.15.26:47588) 2024-04-21T10:29:46.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.189+0000 7fbcdffff700 1 -- 172.21.15.26:0/2909966612 learned_addr learned my addr 172.21.15.26:0/2909966612 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:46.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.189+0000 7fbce50e7700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce0107970 0x7fbce0116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:46.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.190+0000 7fbce50e7700 1 -- 172.21.15.26:0/2909966612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbce006ac90 msgr2=0x7fbce0073430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:46.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.190+0000 7fbce50e7700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbce006ac90 0x7fbce0073430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:46.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.190+0000 7fbce50e7700 1 -- 172.21.15.26:0/2909966612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbce0105480 msgr2=0x7fbce0073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:46.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.190+0000 7fbce50e7700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbce0105480 0x7fbce0073970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:46.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.190+0000 7fbce50e7700 1 -- 172.21.15.26:0/2909966612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce010e0d0 con 0x7fbce0107970 2024-04-21T10:29:46.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.190+0000 7fbce50e7700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce0107970 0x7fbce0116260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800ab40 tx=0x7fbcd800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:46.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.190+0000 7fbcddffb700 1 -- 172.21.15.26:0/2909966612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8017020 con 0x7fbce0107970 2024-04-21T10:29:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.191+0000 7fbcddffb700 1 -- 172.21.15.26:0/2909966612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd8004bb0 con 0x7fbce0107970 2024-04-21T10:29:46.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.191+0000 7fbcddffb700 1 -- 172.21.15.26:0/2909966612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8005310 con 0x7fbce0107970 2024-04-21T10:29:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.192+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2909966612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce010e3c0 con 0x7fbce0107970 2024-04-21T10:29:46.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.192+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2909966612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce010e8b0 con 0x7fbce0107970 2024-04-21T10:29:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.196+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2909966612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbccc001030 con 0x7fbce0107970 2024-04-21T10:29:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.197+0000 7fbcddffb700 1 -- 172.21.15.26:0/2909966612 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbcd800c640 con 0x7fbce0107970 2024-04-21T10:29:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.198+0000 7fbcddffb700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbcc80a3ed0 0x7fbcc80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.198+0000 7fbcddffb700 1 -- 172.21.15.26:0/2909966612 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fbcd80cf920 con 0x7fbce0107970 2024-04-21T10:29:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.198+0000 7fbcddffb700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108] conn(0x7fbcc80ad6b0 0x7fbcc80afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:46.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.199+0000 7fbcddffb700 1 -- 172.21.15.26:0/2909966612 --> [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbcc80b0180 con 0x7fbcc80ad6b0 2024-04-21T10:29:46.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.199+0000 7fbcddffb700 1 -- 172.21.15.26:0/2909966612 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fbcd80d42a0 con 0x7fbce0107970 2024-04-21T10:29:46.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.199+0000 7fbce48e6700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbcc80a3ed0 0x7fbcc80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:46.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.199+0000 7fbcdffff700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108] conn(0x7fbcc80ad6b0 0x7fbcc80afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:46.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.200+0000 7fbcdffff700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108] conn(0x7fbcc80ad6b0 0x7fbcc80afad0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:46.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.200+0000 7fbce48e6700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbcc80a3ed0 0x7fbcc80a6390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400a9b0 tx=0x7fbcd4005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:46.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.201+0000 7fbcddffb700 1 -- 172.21.15.26:0/2909966612 <== osd.0 v2:172.21.15.26:6802/3221939108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcc80b0180 con 0x7fbcc80ad6b0 2024-04-21T10:29:46.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.231+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2909966612 --> [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbccc002e60 con 0x7fbcc80ad6b0 2024-04-21T10:29:46.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.232+0000 7fbcddffb700 1 -- 172.21.15.26:0/2909966612 <== osd.0 v2:172.21.15.26:6802/3221939108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbccc002e60 con 0x7fbcc80ad6b0 2024-04-21T10:29:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.233+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108] conn(0x7fbcc80ad6b0 msgr2=0x7fbcc80afad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.233+0000 7fbce6b4a700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108] conn(0x7fbcc80ad6b0 0x7fbcc80afad0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.233+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbcc80a3ed0 msgr2=0x7fbcc80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.233+0000 7fbce6b4a700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbcc80a3ed0 0x7fbcc80a6390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400a9b0 tx=0x7fbcd4005c90 comp rx=0 tx=0).stop 2024-04-21T10:29:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.233+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce0107970 msgr2=0x7fbce0116260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.233+0000 7fbce6b4a700 1 --2- 172.21.15.26:0/2909966612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce0107970 0x7fbce0116260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800ab40 tx=0x7fbcd800ae50 comp rx=0 tx=0).stop 2024-04-21T10:29:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.233+0000 7fbce50e7700 1 -- 172.21.15.26:0/2909966612 reap_dead start 2024-04-21T10:29:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.233+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2909966612 shutdown_connections 2024-04-21T10:29:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.233+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2909966612 >> 172.21.15.26:0/2909966612 conn(0x7fbce00788f0 msgr2=0x7fbce01002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:46.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.234+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2909966612 shutdown_connections 2024-04-21T10:29:46.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:46.234+0000 7fbce6b4a700 1 -- 172.21.15.26:0/2909966612 wait complete. 2024-04-21T10:29:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:47.503 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:47 smithi026 ceph-mon[27965]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:29:47.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:47 smithi190 ceph-mon[31423]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:29:47.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:47 smithi169 ceph-mon[31469]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:29:48.135 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.401+0000 7f20500aa700 1 -- 172.21.15.26:0/2085144608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2048108a60 msgr2=0x7f2048108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.401+0000 7f20500aa700 1 --2- 172.21.15.26:0/2085144608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2048108a60 0x7f2048108e40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f204000b0a0 tx=0x7f204000b3b0 comp rx=0 tx=0).stop 2024-04-21T10:29:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.401+0000 7f20500aa700 1 -- 172.21.15.26:0/2085144608 shutdown_connections 2024-04-21T10:29:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.401+0000 7f20500aa700 1 --2- 172.21.15.26:0/2085144608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f204806bbb0 0x7f204810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.401+0000 7f20500aa700 1 --2- 172.21.15.26:0/2085144608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2048108a60 0x7f2048108e40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.401+0000 7f20500aa700 1 --2- 172.21.15.26:0/2085144608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f204806b1f0 0x7f204806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.401+0000 7f20500aa700 1 -- 172.21.15.26:0/2085144608 >> 172.21.15.26:0/2085144608 conn(0x7f20480789e0 msgr2=0x7f2048078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.401+0000 7f20500aa700 1 -- 172.21.15.26:0/2085144608 shutdown_connections 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.401+0000 7f20500aa700 1 -- 172.21.15.26:0/2085144608 wait complete. 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f20500aa700 1 Processor -- start 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f20500aa700 1 -- start start 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f20500aa700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f204806b1f0 0x7f2048199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f20500aa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204806bbb0 0x7f20481997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f20500aa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2048108a60 0x7f204819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f20500aa700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2048112cf0 con 0x7f204806bbb0 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f20500aa700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2048112b70 con 0x7f204806b1f0 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f20500aa700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2048112e70 con 0x7f2048108a60 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f204d645700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204806bbb0 0x7f20481997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f204d645700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204806bbb0 0x7f20481997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50662/0 (socket says 172.21.15.26:50662) 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f204d645700 1 -- 172.21.15.26:0/3902931970 learned_addr learned my addr 172.21.15.26:0/3902931970 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f204de46700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f204806b1f0 0x7f2048199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f204e647700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2048108a60 0x7f204819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f204d645700 1 -- 172.21.15.26:0/3902931970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2048108a60 msgr2=0x7f204819dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f204d645700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2048108a60 0x7f204819dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f204d645700 1 -- 172.21.15.26:0/3902931970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f204806b1f0 msgr2=0x7f2048199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f204d645700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f204806b1f0 0x7f2048199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.402+0000 7f204d645700 1 -- 172.21.15.26:0/3902931970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204819e220 con 0x7f204806bbb0 2024-04-21T10:29:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.403+0000 7f204d645700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204806bbb0 0x7f20481997a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f203c00b530 tx=0x7f203c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.403+0000 7f2046ffd700 1 -- 172.21.15.26:0/3902931970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c015020 con 0x7f204806bbb0 2024-04-21T10:29:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.403+0000 7f20500aa700 1 -- 172.21.15.26:0/3902931970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204819e510 con 0x7f204806bbb0 2024-04-21T10:29:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.403+0000 7f2046ffd700 1 -- 172.21.15.26:0/3902931970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f203c011ce0 con 0x7f204806bbb0 2024-04-21T10:29:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.403+0000 7f2046ffd700 1 -- 172.21.15.26:0/3902931970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c010e40 con 0x7f204806bbb0 2024-04-21T10:29:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.403+0000 7f20500aa700 1 -- 172.21.15.26:0/3902931970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20481a5d90 con 0x7f204806bbb0 2024-04-21T10:29:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.404+0000 7f20500aa700 1 -- 172.21.15.26:0/3902931970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20480631d0 con 0x7f204806bbb0 2024-04-21T10:29:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.405+0000 7f2046ffd700 1 -- 172.21.15.26:0/3902931970 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f203c018020 con 0x7f204806bbb0 2024-04-21T10:29:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.406+0000 7f2046ffd700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f20300a3cd0 0x7f20300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.406+0000 7f204de46700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f20300a3cd0 0x7f20300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.406+0000 7f2046ffd700 1 -- 172.21.15.26:0/3902931970 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f203c0d0200 con 0x7f204806bbb0 2024-04-21T10:29:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.406+0000 7f2046ffd700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914] conn(0x7f20300ad4b0 0x7f20300af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.406+0000 7f2046ffd700 1 -- 172.21.15.26:0/3902931970 --> [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20300aff80 con 0x7f20300ad4b0 2024-04-21T10:29:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.406+0000 7f2046ffd700 1 -- 172.21.15.26:0/3902931970 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f203c0d06e0 con 0x7f204806bbb0 2024-04-21T10:29:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.406+0000 7f204de46700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f20300a3cd0 0x7f20300a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2040009e60 tx=0x7f204000b7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.406+0000 7f204e647700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914] conn(0x7f20300ad4b0 0x7f20300af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.407+0000 7f204e647700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914] conn(0x7f20300ad4b0 0x7f20300af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.408+0000 7f2046ffd700 1 -- 172.21.15.26:0/3902931970 <== osd.5 v2:172.21.15.190:6808/427467914 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20300aff80 con 0x7f20300ad4b0 2024-04-21T10:29:48.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.424+0000 7f20500aa700 1 -- 172.21.15.26:0/3902931970 --> [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2048063650 con 0x7f20300ad4b0 2024-04-21T10:29:48.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.425+0000 7f2046ffd700 1 -- 172.21.15.26:0/3902931970 <== osd.5 v2:172.21.15.190:6808/427467914 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2048063650 con 0x7f20300ad4b0 2024-04-21T10:29:48.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.425+0000 7f20500aa700 1 -- 172.21.15.26:0/3902931970 >> [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914] conn(0x7f20300ad4b0 msgr2=0x7f20300af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:48.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.425+0000 7f20500aa700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914] conn(0x7f20300ad4b0 0x7f20300af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:48.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.425+0000 7f20500aa700 1 -- 172.21.15.26:0/3902931970 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f20300a3cd0 msgr2=0x7f20300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:48.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.425+0000 7f20500aa700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f20300a3cd0 0x7f20300a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2040009e60 tx=0x7f204000b7f0 comp rx=0 tx=0).stop 2024-04-21T10:29:48.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.425+0000 7f20500aa700 1 -- 172.21.15.26:0/3902931970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204806bbb0 msgr2=0x7f20481997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:48.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.425+0000 7f20500aa700 1 --2- 172.21.15.26:0/3902931970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f204806bbb0 0x7f20481997a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f203c00b530 tx=0x7f203c00b840 comp rx=0 tx=0).stop 2024-04-21T10:29:48.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.426+0000 7f204e647700 1 -- 172.21.15.26:0/3902931970 reap_dead start 2024-04-21T10:29:48.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.426+0000 7f20500aa700 1 -- 172.21.15.26:0/3902931970 shutdown_connections 2024-04-21T10:29:48.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.426+0000 7f20500aa700 1 -- 172.21.15.26:0/3902931970 >> 172.21.15.26:0/3902931970 conn(0x7f20480789e0 msgr2=0x7f20480ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:48.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.426+0000 7f20500aa700 1 -- 172.21.15.26:0/3902931970 shutdown_connections 2024-04-21T10:29:48.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:48.426+0000 7f20500aa700 1 -- 172.21.15.26:0/3902931970 wait complete. 2024-04-21T10:29:49.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:49 smithi026 ceph-mon[27965]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:29:49.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:49 smithi190 ceph-mon[31423]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:29:49.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:49 smithi169 ceph-mon[31469]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:29:50.123 INFO:teuthology.orchestra.run.smithi026.stdout:34359738444 2024-04-21T10:29:50.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-21T10:29:51.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:51 smithi190 ceph-mon[31423]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-21T10:29:51.550 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:51 smithi026 ceph-mon[27965]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-21T10:29:51.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:51 smithi169 ceph-mon[31469]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-21T10:29:51.752 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:29:52.168 INFO:teuthology.orchestra.run.smithi026.stdout:171798691894 2024-04-21T10:29:52.168 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-21T10:29:53.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:53 smithi026 ceph-mon[27965]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:29:53.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:53 smithi190 ceph-mon[31423]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:29:53.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:53 smithi169 ceph-mon[31469]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:29:54.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.003+0000 7f79ea65d700 1 -- 172.21.15.26:0/4245041957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e4108a50 msgr2=0x7f79e4108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:54.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.003+0000 7f79ea65d700 1 --2- 172.21.15.26:0/4245041957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e4108a50 0x7f79e4108e30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009990 tx=0x7f79d4009ca0 comp rx=0 tx=0).stop 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.003+0000 7f79ea65d700 1 -- 172.21.15.26:0/4245041957 shutdown_connections 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.003+0000 7f79ea65d700 1 --2- 172.21.15.26:0/4245041957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e4103450 0x7f79e410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.003+0000 7f79ea65d700 1 --2- 172.21.15.26:0/4245041957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e4108a50 0x7f79e4108e30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.003+0000 7f79ea65d700 1 --2- 172.21.15.26:0/4245041957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79e4102a90 0x7f79e4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.003+0000 7f79ea65d700 1 -- 172.21.15.26:0/4245041957 >> 172.21.15.26:0/4245041957 conn(0x7f79e4079ba0 msgr2=0x7f79e4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.003+0000 7f79ea65d700 1 -- 172.21.15.26:0/4245041957 shutdown_connections 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.003+0000 7f79ea65d700 1 -- 172.21.15.26:0/4245041957 wait complete. 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.004+0000 7f79ea65d700 1 Processor -- start 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.004+0000 7f79ea65d700 1 -- start start 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.004+0000 7f79ea65d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79e4102a90 0x7f79e4199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.004+0000 7f79ea65d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e4103450 0x7f79e4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.004+0000 7f79ea65d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e4108a50 0x7f79e419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.004+0000 7f79ea65d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79e4112c60 con 0x7f79e4108a50 2024-04-21T10:29:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.004+0000 7f79ea65d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79e4112ae0 con 0x7f79e4103450 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.004+0000 7f79ea65d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f79e4112de0 con 0x7f79e4102a90 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.004+0000 7f79e8bfa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e4108a50 0x7f79e419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.004+0000 7f79e3fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79e4102a90 0x7f79e4199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.004+0000 7f79e8bfa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e4108a50 0x7f79e419d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50676/0 (socket says 172.21.15.26:50676) 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.004+0000 7f79e8bfa700 1 -- 172.21.15.26:0/2874967031 learned_addr learned my addr 172.21.15.26:0/2874967031 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.005+0000 7f79e37fe700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e4103450 0x7f79e4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.005+0000 7f79e8bfa700 1 -- 172.21.15.26:0/2874967031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79e4102a90 msgr2=0x7f79e4199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.005+0000 7f79e8bfa700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79e4102a90 0x7f79e4199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.005+0000 7f79e8bfa700 1 -- 172.21.15.26:0/2874967031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e4103450 msgr2=0x7f79e4199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.005+0000 7f79e8bfa700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e4103450 0x7f79e4199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.005+0000 7f79e8bfa700 1 -- 172.21.15.26:0/2874967031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e419e120 con 0x7f79e4108a50 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.005+0000 7f79e8bfa700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e4108a50 0x7f79e419d9e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f79dc007d70 tx=0x7f79dc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.005+0000 7f79e17fa700 1 -- 172.21.15.26:0/2874967031 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc016020 con 0x7f79e4108a50 2024-04-21T10:29:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.005+0000 7f79ea65d700 1 -- 172.21.15.26:0/2874967031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e419e410 con 0x7f79e4108a50 2024-04-21T10:29:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.005+0000 7f79e17fa700 1 -- 172.21.15.26:0/2874967031 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79dc00ad40 con 0x7f79e4108a50 2024-04-21T10:29:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.005+0000 7f79e17fa700 1 -- 172.21.15.26:0/2874967031 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc0115e0 con 0x7f79e4108a50 2024-04-21T10:29:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.005+0000 7f79ea65d700 1 -- 172.21.15.26:0/2874967031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e41a5c90 con 0x7f79e4108a50 2024-04-21T10:29:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.006+0000 7f79ea65d700 1 -- 172.21.15.26:0/2874967031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e4069730 con 0x7f79e4108a50 2024-04-21T10:29:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.008+0000 7f79e17fa700 1 -- 172.21.15.26:0/2874967031 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f79dc019020 con 0x7f79e4108a50 2024-04-21T10:29:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.008+0000 7f79e17fa700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f79cc0a3cd0 0x7f79cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.008+0000 7f79e3fff700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f79cc0a3cd0 0x7f79cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.009+0000 7f79e17fa700 1 -- 172.21.15.26:0/2874967031 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f79dc0d0600 con 0x7f79e4108a50 2024-04-21T10:29:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.009+0000 7f79e3fff700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f79cc0a3cd0 0x7f79cc0a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f79d400b5c0 tx=0x7f79d4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.012+0000 7f79e17fa700 1 -- 172.21.15.26:0/2874967031 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79dc020020 con 0x7f79e4108a50 2024-04-21T10:29:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.231+0000 7f79ea65d700 1 -- 172.21.15.26:0/2874967031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f79e402df80 con 0x7f79e4108a50 2024-04-21T10:29:54.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.231+0000 7f79e17fa700 1 -- 172.21.15.26:0/2874967031 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f79dc09d140 con 0x7f79e4108a50 2024-04-21T10:29:54.231 INFO:teuthology.orchestra.run.smithi026.stdout:227633266731 2024-04-21T10:29:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.233+0000 7f79ea65d700 1 -- 172.21.15.26:0/2874967031 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f79cc0a3cd0 msgr2=0x7f79cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.233+0000 7f79ea65d700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f79cc0a3cd0 0x7f79cc0a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f79d400b5c0 tx=0x7f79d4005960 comp rx=0 tx=0).stop 2024-04-21T10:29:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.233+0000 7f79ea65d700 1 -- 172.21.15.26:0/2874967031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e4108a50 msgr2=0x7f79e419d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.233+0000 7f79ea65d700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e4108a50 0x7f79e419d9e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f79dc007d70 tx=0x7f79dc00a340 comp rx=0 tx=0).stop 2024-04-21T10:29:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.233+0000 7f79ea65d700 1 -- 172.21.15.26:0/2874967031 shutdown_connections 2024-04-21T10:29:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.233+0000 7f79ea65d700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f79cc0a3cd0 0x7f79cc0a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.233+0000 7f79ea65d700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79e4108a50 0x7f79e419d9e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.233+0000 7f79ea65d700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79e4102a90 0x7f79e4199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.233+0000 7f79ea65d700 1 --2- 172.21.15.26:0/2874967031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e4103450 0x7f79e4199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.233+0000 7f79ea65d700 1 -- 172.21.15.26:0/2874967031 >> 172.21.15.26:0/2874967031 conn(0x7f79e4079ba0 msgr2=0x7f79e41016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.233+0000 7f79ea65d700 1 -- 172.21.15.26:0/2874967031 shutdown_connections 2024-04-21T10:29:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:54.233+0000 7f79ea65d700 1 -- 172.21.15.26:0/2874967031 wait complete. 2024-04-21T10:29:54.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:54 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2874967031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T10:29:54.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:54 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2874967031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T10:29:54.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:54 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2874967031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T10:29:55.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:55 smithi026 ceph-mon[27965]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:29:55.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:55 smithi190 ceph-mon[31423]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:29:55.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:55 smithi169 ceph-mon[31469]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:29:57.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.278+0000 7f5029b7a700 1 -- 172.21.15.26:0/3306438837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502406b170 msgr2=0x7f502406b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.278+0000 7f5029b7a700 1 --2- 172.21.15.26:0/3306438837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502406b170 0x7f502406b5d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5018009990 tx=0x7f5018009ca0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.278+0000 7f5029b7a700 1 -- 172.21.15.26:0/3306438837 shutdown_connections 2024-04-21T10:29:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.278+0000 7f5029b7a700 1 --2- 172.21.15.26:0/3306438837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024108a30 0x7f5024108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.278+0000 7f5029b7a700 1 --2- 172.21.15.26:0/3306438837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f502406bb10 0x7f502410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.278+0000 7f5029b7a700 1 --2- 172.21.15.26:0/3306438837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502406b170 0x7f502406b5d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.278+0000 7f5029b7a700 1 -- 172.21.15.26:0/3306438837 >> 172.21.15.26:0/3306438837 conn(0x7f50240788f0 msgr2=0x7f5024078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.279+0000 7f5029b7a700 1 -- 172.21.15.26:0/3306438837 shutdown_connections 2024-04-21T10:29:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.279+0000 7f5029b7a700 1 -- 172.21.15.26:0/3306438837 wait complete. 2024-04-21T10:29:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.279+0000 7f5029b7a700 1 Processor -- start 2024-04-21T10:29:57.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.279+0000 7f5029b7a700 1 -- start start 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5029b7a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f502406b170 0x7f502419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5029b7a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502406bb10 0x7f502419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5029b7a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024108a30 0x7f502419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5029b7a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5024112c10 con 0x7f502406bb10 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5029b7a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5024112a90 con 0x7f5024108a30 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5029b7a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5024112d90 con 0x7f502406b170 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5022ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502406bb10 0x7f502419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5022ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502406bb10 0x7f502419b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43648/0 (socket says 172.21.15.26:43648) 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f50237fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f502406b170 0x7f502419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5022ffd700 1 -- 172.21.15.26:0/3087861169 learned_addr learned my addr 172.21.15.26:0/3087861169 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5023fff700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024108a30 0x7f502419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5022ffd700 1 -- 172.21.15.26:0/3087861169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f502406b170 msgr2=0x7f502419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:57.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5022ffd700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f502406b170 0x7f502419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5022ffd700 1 -- 172.21.15.26:0/3087861169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024108a30 msgr2=0x7f502419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5022ffd700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024108a30 0x7f502419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5022ffd700 1 -- 172.21.15.26:0/3087861169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50241a02a0 con 0x7f502406bb10 2024-04-21T10:29:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.280+0000 7f5022ffd700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502406bb10 0x7f502419b7d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5018000c00 tx=0x7f50180047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.281+0000 7f5020ff9700 1 -- 172.21.15.26:0/3087861169 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018027070 con 0x7f502406bb10 2024-04-21T10:29:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.281+0000 7f5020ff9700 1 -- 172.21.15.26:0/3087861169 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5018003f20 con 0x7f502406bb10 2024-04-21T10:29:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.281+0000 7f5029b7a700 1 -- 172.21.15.26:0/3087861169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50241a0530 con 0x7f502406bb10 2024-04-21T10:29:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.281+0000 7f5020ff9700 1 -- 172.21.15.26:0/3087861169 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018022ca0 con 0x7f502406bb10 2024-04-21T10:29:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.281+0000 7f5029b7a700 1 -- 172.21.15.26:0/3087861169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50241a7e10 con 0x7f502406bb10 2024-04-21T10:29:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.282+0000 7f5029b7a700 1 -- 172.21.15.26:0/3087861169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5024104170 con 0x7f502406bb10 2024-04-21T10:29:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.283+0000 7f5020ff9700 1 -- 172.21.15.26:0/3087861169 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f501802a070 con 0x7f502406bb10 2024-04-21T10:29:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.284+0000 7f5020ff9700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f500c0a3cd0 0x7f500c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.284+0000 7f5020ff9700 1 -- 172.21.15.26:0/3087861169 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f50180d1240 con 0x7f502406bb10 2024-04-21T10:29:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.284+0000 7f50237fe700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f500c0a3cd0 0x7f500c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:57.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.285+0000 7f50237fe700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f500c0a3cd0 0x7f500c0a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f501400abd0 tx=0x7f501400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:57.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.289+0000 7f5020ff9700 1 -- 172.21.15.26:0/3087861169 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f501805eff0 con 0x7f502406bb10 2024-04-21T10:29:57.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:57 smithi026 ceph-mon[27965]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:29:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.500+0000 7f5029b7a700 1 -- 172.21.15.26:0/3087861169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f502402df80 con 0x7f502406bb10 2024-04-21T10:29:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.500+0000 7f5020ff9700 1 -- 172.21.15.26:0/3087861169 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5018013070 con 0x7f502406bb10 2024-04-21T10:29:57.500 INFO:teuthology.orchestra.run.smithi026.stdout:55834574923 2024-04-21T10:29:57.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.502+0000 7f5029b7a700 1 -- 172.21.15.26:0/3087861169 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f500c0a3cd0 msgr2=0x7f500c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:57.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.502+0000 7f5029b7a700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f500c0a3cd0 0x7f500c0a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f501400abd0 tx=0x7f501400a380 comp rx=0 tx=0).stop 2024-04-21T10:29:57.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.502+0000 7f5029b7a700 1 -- 172.21.15.26:0/3087861169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502406bb10 msgr2=0x7f502419b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:57.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.502+0000 7f5029b7a700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502406bb10 0x7f502419b7d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5018000c00 tx=0x7f50180047b0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.502+0000 7f5029b7a700 1 -- 172.21.15.26:0/3087861169 shutdown_connections 2024-04-21T10:29:57.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.502+0000 7f5029b7a700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f500c0a3cd0 0x7f500c0a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.502+0000 7f5029b7a700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024108a30 0x7f502419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.502+0000 7f5029b7a700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f502406b170 0x7f502419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.502+0000 7f5029b7a700 1 --2- 172.21.15.26:0/3087861169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502406bb10 0x7f502419b7d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.502+0000 7f5029b7a700 1 -- 172.21.15.26:0/3087861169 >> 172.21.15.26:0/3087861169 conn(0x7f50240788f0 msgr2=0x7f502410de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.502+0000 7f5029b7a700 1 -- 172.21.15.26:0/3087861169 shutdown_connections 2024-04-21T10:29:57.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.502+0000 7f5029b7a700 1 -- 172.21.15.26:0/3087861169 wait complete. 2024-04-21T10:29:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.633+0000 7fc419222700 1 -- 172.21.15.26:0/1295654986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc414102ac0 msgr2=0x7fc414102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.633+0000 7fc419222700 1 --2- 172.21.15.26:0/1295654986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc414102ac0 0x7fc414102f40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc408009990 tx=0x7fc408009ca0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.633+0000 7fc419222700 1 -- 172.21.15.26:0/1295654986 shutdown_connections 2024-04-21T10:29:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.633+0000 7fc419222700 1 --2- 172.21.15.26:0/1295654986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc414108ab0 0x7fc414108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.633+0000 7fc419222700 1 --2- 172.21.15.26:0/1295654986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414103480 0x7fc41410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.633+0000 7fc419222700 1 --2- 172.21.15.26:0/1295654986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc414102ac0 0x7fc414102f40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.633+0000 7fc419222700 1 -- 172.21.15.26:0/1295654986 >> 172.21.15.26:0/1295654986 conn(0x7fc414079bf0 msgr2=0x7fc41407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.633+0000 7fc419222700 1 -- 172.21.15.26:0/1295654986 shutdown_connections 2024-04-21T10:29:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.633+0000 7fc419222700 1 -- 172.21.15.26:0/1295654986 wait complete. 2024-04-21T10:29:57.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc419222700 1 Processor -- start 2024-04-21T10:29:57.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc419222700 1 -- start start 2024-04-21T10:29:57.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc419222700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc414102ac0 0x7fc41419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc419222700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414103480 0x7fc41419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc419222700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc414108ab0 0x7fc41419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc419222700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc414112cf0 con 0x7fc414102ac0 2024-04-21T10:29:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc419222700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc414112b70 con 0x7fc414108ab0 2024-04-21T10:29:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc419222700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc414112e70 con 0x7fc414103480 2024-04-21T10:29:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc412d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc414102ac0 0x7fc41419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc412d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc414102ac0 0x7fc41419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43668/0 (socket says 172.21.15.26:43668) 2024-04-21T10:29:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc412d9d700 1 -- 172.21.15.26:0/1323922236 learned_addr learned my addr 172.21.15.26:0/1323922236 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc41259c700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414103480 0x7fc41419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.634+0000 7fc41359e700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc414108ab0 0x7fc41419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.635+0000 7fc412d9d700 1 -- 172.21.15.26:0/1323922236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414103480 msgr2=0x7fc41419b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:57.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.635+0000 7fc412d9d700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414103480 0x7fc41419b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.635+0000 7fc412d9d700 1 -- 172.21.15.26:0/1323922236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc414108ab0 msgr2=0x7fc41419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.635+0000 7fc412d9d700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc414108ab0 0x7fc41419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.635+0000 7fc412d9d700 1 -- 172.21.15.26:0/1323922236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4141a0450 con 0x7fc414102ac0 2024-04-21T10:29:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.635+0000 7fc412d9d700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc414102ac0 0x7fc41419b440 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc40400b530 tx=0x7fc40400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.635+0000 7fc3fbfff700 1 -- 172.21.15.26:0/1323922236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404015020 con 0x7fc414102ac0 2024-04-21T10:29:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.635+0000 7fc3fbfff700 1 -- 172.21.15.26:0/1323922236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc404011ce0 con 0x7fc414102ac0 2024-04-21T10:29:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.635+0000 7fc419222700 1 -- 172.21.15.26:0/1323922236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4141a0740 con 0x7fc414102ac0 2024-04-21T10:29:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.635+0000 7fc3fbfff700 1 -- 172.21.15.26:0/1323922236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404010d20 con 0x7fc414102ac0 2024-04-21T10:29:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.635+0000 7fc419222700 1 -- 172.21.15.26:0/1323922236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4141a7fc0 con 0x7fc414102ac0 2024-04-21T10:29:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.636+0000 7fc419222700 1 -- 172.21.15.26:0/1323922236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc414069730 con 0x7fc414102ac0 2024-04-21T10:29:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.637+0000 7fc3fbfff700 1 -- 172.21.15.26:0/1323922236 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc404018020 con 0x7fc414102ac0 2024-04-21T10:29:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.638+0000 7fc3fbfff700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc3fc0a3d50 0x7fc3fc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.638+0000 7fc41259c700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc3fc0a3d50 0x7fc3fc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.638+0000 7fc3fbfff700 1 -- 172.21.15.26:0/1323922236 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fc4040cfa60 con 0x7fc414102ac0 2024-04-21T10:29:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.639+0000 7fc41259c700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc3fc0a3d50 0x7fc3fc0a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc408006010 tx=0x7fc40801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:57.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.643+0000 7fc3fbfff700 1 -- 172.21.15.26:0/1323922236 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc40401f020 con 0x7fc414102ac0 2024-04-21T10:29:57.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:57 smithi169 ceph-mon[31469]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:29:57.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:57 smithi190 ceph-mon[31423]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:29:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.860+0000 7fc419222700 1 -- 172.21.15.26:0/1323922236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc4141a81f0 con 0x7fc414102ac0 2024-04-21T10:29:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.860+0000 7fc3fbfff700 1 -- 172.21.15.26:0/1323922236 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc40409c5a0 con 0x7fc414102ac0 2024-04-21T10:29:57.860 INFO:teuthology.orchestra.run.smithi026.stdout:77309411397 2024-04-21T10:29:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.864+0000 7fc419222700 1 -- 172.21.15.26:0/1323922236 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc3fc0a3d50 msgr2=0x7fc3fc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.864+0000 7fc419222700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc3fc0a3d50 0x7fc3fc0a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc408006010 tx=0x7fc40801a040 comp rx=0 tx=0).stop 2024-04-21T10:29:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.864+0000 7fc419222700 1 -- 172.21.15.26:0/1323922236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc414102ac0 msgr2=0x7fc41419b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.864+0000 7fc419222700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc414102ac0 0x7fc41419b440 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc40400b530 tx=0x7fc40400b8f0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.864+0000 7fc419222700 1 -- 172.21.15.26:0/1323922236 shutdown_connections 2024-04-21T10:29:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.864+0000 7fc419222700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc3fc0a3d50 0x7fc3fc0a6210 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.864+0000 7fc419222700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc414108ab0 0x7fc41419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.864+0000 7fc419222700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc414102ac0 0x7fc41419b440 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.864+0000 7fc419222700 1 --2- 172.21.15.26:0/1323922236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc414103480 0x7fc41419b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.864+0000 7fc419222700 1 -- 172.21.15.26:0/1323922236 >> 172.21.15.26:0/1323922236 conn(0x7fc414079bf0 msgr2=0x7fc41410dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.864+0000 7fc419222700 1 -- 172.21.15.26:0/1323922236 shutdown_connections 2024-04-21T10:29:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:57.864+0000 7fc419222700 1 -- 172.21.15.26:0/1323922236 wait complete. 2024-04-21T10:29:58.223 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574923 for osd.1 2024-04-21T10:29:58.224 DEBUG:teuthology.parallel:result is None 2024-04-21T10:29:58.629 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:58 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3087861169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T10:29:58.630 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:58 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1323922236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T10:29:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:58 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3087861169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T10:29:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:58 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1323922236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T10:29:58.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:58 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3087861169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T10:29:58.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:58 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1323922236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T10:29:58.804 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411397 for osd.2 2024-04-21T10:29:58.804 DEBUG:teuthology.parallel:result is None 2024-04-21T10:29:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:29:59 smithi026 ceph-mon[27965]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:29:59.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.628+0000 7fe315a8e700 1 -- 172.21.15.26:0/3935224347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe310101da0 msgr2=0x7fe31010f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:59.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.628+0000 7fe315a8e700 1 --2- 172.21.15.26:0/3935224347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe310101da0 0x7fe31010f810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe30800b0a0 tx=0x7fe30800b3b0 comp rx=0 tx=0).stop 2024-04-21T10:29:59.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.628+0000 7fe315a8e700 1 -- 172.21.15.26:0/3935224347 shutdown_connections 2024-04-21T10:29:59.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.628+0000 7fe315a8e700 1 --2- 172.21.15.26:0/3935224347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe310107b00 0x7fe310107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:59.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.628+0000 7fe315a8e700 1 --2- 172.21.15.26:0/3935224347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe310101da0 0x7fe31010f810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:59.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.628+0000 7fe315a8e700 1 --2- 172.21.15.26:0/3935224347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3101013e0 0x7fe310101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:59.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.628+0000 7fe315a8e700 1 -- 172.21.15.26:0/3935224347 >> 172.21.15.26:0/3935224347 conn(0x7fe310076c60 msgr2=0x7fe310077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:59.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.628+0000 7fe315a8e700 1 -- 172.21.15.26:0/3935224347 shutdown_connections 2024-04-21T10:29:59.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.628+0000 7fe315a8e700 1 -- 172.21.15.26:0/3935224347 wait complete. 2024-04-21T10:29:59.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.628+0000 7fe315a8e700 1 Processor -- start 2024-04-21T10:29:59.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe315a8e700 1 -- start start 2024-04-21T10:29:59.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe315a8e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3101013e0 0x7fe310199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe315a8e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe310101da0 0x7fe3101997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe315a8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe310107b00 0x7fe31019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe315a8e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe310112d10 con 0x7fe310107b00 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe315a8e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe310112b90 con 0x7fe3101013e0 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe315a8e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe310112e90 con 0x7fe310101da0 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe30ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe310107b00 0x7fe31019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe30ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe310107b00 0x7fe31019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43684/0 (socket says 172.21.15.26:43684) 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe30ffff700 1 -- 172.21.15.26:0/624046735 learned_addr learned my addr 172.21.15.26:0/624046735 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe30f7fe700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3101013e0 0x7fe310199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe30effd700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe310101da0 0x7fe3101997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe30ffff700 1 -- 172.21.15.26:0/624046735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe310101da0 msgr2=0x7fe3101997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe30ffff700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe310101da0 0x7fe3101997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe30ffff700 1 -- 172.21.15.26:0/624046735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3101013e0 msgr2=0x7fe310199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:59.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe30ffff700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3101013e0 0x7fe310199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:59.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe30ffff700 1 -- 172.21.15.26:0/624046735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31019e220 con 0x7fe310107b00 2024-04-21T10:29:59.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe30f7fe700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3101013e0 0x7fe310199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:29:59.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.629+0000 7fe30effd700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe310101da0 0x7fe3101997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:29:59.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.630+0000 7fe30ffff700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe310107b00 0x7fe31019dae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3080062a0 tx=0x7fe308004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:59.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.630+0000 7fe30cff9700 1 -- 172.21.15.26:0/624046735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308021070 con 0x7fe310107b00 2024-04-21T10:29:59.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.630+0000 7fe30cff9700 1 -- 172.21.15.26:0/624046735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe308007d00 con 0x7fe310107b00 2024-04-21T10:29:59.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.630+0000 7fe315a8e700 1 -- 172.21.15.26:0/624046735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31019e4b0 con 0x7fe310107b00 2024-04-21T10:29:59.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.630+0000 7fe30cff9700 1 -- 172.21.15.26:0/624046735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30801d450 con 0x7fe310107b00 2024-04-21T10:29:59.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.630+0000 7fe315a8e700 1 -- 172.21.15.26:0/624046735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3101a5d90 con 0x7fe310107b00 2024-04-21T10:29:59.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.631+0000 7fe315a8e700 1 -- 172.21.15.26:0/624046735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f0005330 con 0x7fe310107b00 2024-04-21T10:29:59.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.632+0000 7fe30cff9700 1 -- 172.21.15.26:0/624046735 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe308004020 con 0x7fe310107b00 2024-04-21T10:29:59.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.632+0000 7fe30cff9700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe2fc0a3d50 0x7fe2fc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:29:59.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.633+0000 7fe30f7fe700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe2fc0a3d50 0x7fe2fc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:29:59.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.633+0000 7fe30cff9700 1 -- 172.21.15.26:0/624046735 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fe3080d1550 con 0x7fe310107b00 2024-04-21T10:29:59.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.633+0000 7fe30f7fe700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe2fc0a3d50 0x7fe2fc0a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe3100781e0 tx=0x7fe2f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:29:59.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.637+0000 7fe30cff9700 1 -- 172.21.15.26:0/624046735 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe30809df10 con 0x7fe310107b00 2024-04-21T10:29:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:29:59 smithi169 ceph-mon[31469]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:29:59.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:29:59 smithi190 ceph-mon[31423]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:29:59.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.848+0000 7fe315a8e700 1 -- 172.21.15.26:0/624046735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe2f00058f0 con 0x7fe310107b00 2024-04-21T10:29:59.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.849+0000 7fe30cff9700 1 -- 172.21.15.26:0/624046735 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe30809d810 con 0x7fe310107b00 2024-04-21T10:29:59.849 INFO:teuthology.orchestra.run.smithi026.stdout:261993005096 2024-04-21T10:29:59.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.851+0000 7fe315a8e700 1 -- 172.21.15.26:0/624046735 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe2fc0a3d50 msgr2=0x7fe2fc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:59.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.851+0000 7fe315a8e700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe2fc0a3d50 0x7fe2fc0a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe3100781e0 tx=0x7fe2f8008040 comp rx=0 tx=0).stop 2024-04-21T10:29:59.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.851+0000 7fe315a8e700 1 -- 172.21.15.26:0/624046735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe310107b00 msgr2=0x7fe31019dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:29:59.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.851+0000 7fe315a8e700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe310107b00 0x7fe31019dae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3080062a0 tx=0x7fe308004ed0 comp rx=0 tx=0).stop 2024-04-21T10:29:59.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.851+0000 7fe315a8e700 1 -- 172.21.15.26:0/624046735 shutdown_connections 2024-04-21T10:29:59.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.851+0000 7fe315a8e700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe2fc0a3d50 0x7fe2fc0a6210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.851+0000 7fe315a8e700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3101013e0 0x7fe310199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.851+0000 7fe315a8e700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe310107b00 0x7fe31019dae0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.851+0000 7fe315a8e700 1 --2- 172.21.15.26:0/624046735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe310101da0 0x7fe3101997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:29:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.851+0000 7fe315a8e700 1 -- 172.21.15.26:0/624046735 >> 172.21.15.26:0/624046735 conn(0x7fe310076c60 msgr2=0x7fe3101000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:29:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.851+0000 7fe315a8e700 1 -- 172.21.15.26:0/624046735 shutdown_connections 2024-04-21T10:29:59.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:29:59.851+0000 7fe315a8e700 1 -- 172.21.15.26:0/624046735 wait complete. 2024-04-21T10:30:00.630 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:00 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/624046735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T10:30:00.630 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:00 smithi026 ceph-mon[27965]: overall HEALTH_OK 2024-04-21T10:30:00.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:00 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/624046735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T10:30:00.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:00 smithi169 ceph-mon[31469]: overall HEALTH_OK 2024-04-21T10:30:00.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:00 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/624046735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T10:30:00.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:00 smithi190 ceph-mon[31423]: overall HEALTH_OK 2024-04-21T10:30:01.262 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005093 got 261993005096 for osd.8 2024-04-21T10:30:01.262 DEBUG:teuthology.parallel:result is None 2024-04-21T10:30:01.668 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:01 smithi026 ceph-mon[27965]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:01.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:01 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:30:01.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:01 smithi169 ceph-mon[31469]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:01.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:01 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:30:01.734 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:01 smithi190 ceph-mon[31423]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:01.734 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:01 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:30:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.876+0000 7f80857db700 1 -- 172.21.15.26:0/2940584356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101940 msgr2=0x7f8080101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.876+0000 7f80857db700 1 --2- 172.21.15.26:0/2940584356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101940 0x7f8080101dc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8074009990 tx=0x7f8074009ca0 comp rx=0 tx=0).stop 2024-04-21T10:30:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.876+0000 7f80857db700 1 -- 172.21.15.26:0/2940584356 shutdown_connections 2024-04-21T10:30:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.876+0000 7f80857db700 1 --2- 172.21.15.26:0/2940584356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8080107990 0x7f8080107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.876+0000 7f80857db700 1 --2- 172.21.15.26:0/2940584356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8080102300 0x7f808010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.876+0000 7f80857db700 1 --2- 172.21.15.26:0/2940584356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080101940 0x7f8080101dc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.876+0000 7f80857db700 1 -- 172.21.15.26:0/2940584356 >> 172.21.15.26:0/2940584356 conn(0x7f80800788f0 msgr2=0x7f8080078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.876+0000 7f80857db700 1 -- 172.21.15.26:0/2940584356 shutdown_connections 2024-04-21T10:30:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.876+0000 7f80857db700 1 -- 172.21.15.26:0/2940584356 wait complete. 2024-04-21T10:30:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.877+0000 7f80857db700 1 Processor -- start 2024-04-21T10:30:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.877+0000 7f80857db700 1 -- start start 2024-04-21T10:30:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f80857db700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8080101940 0x7f8080198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f80857db700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080102300 0x7f8080198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f80857db700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8080107990 0x7f808019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f80857db700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8080111af0 con 0x7f8080102300 2024-04-21T10:30:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f80857db700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8080111970 con 0x7f8080101940 2024-04-21T10:30:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f80857db700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8080111c70 con 0x7f8080107990 2024-04-21T10:30:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f807e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080102300 0x7f8080198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f807e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080102300 0x7f8080198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43710/0 (socket says 172.21.15.26:43710) 2024-04-21T10:30:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f807e7fc700 1 -- 172.21.15.26:0/3725096013 learned_addr learned my addr 172.21.15.26:0/3725096013 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f807f7fe700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8080107990 0x7f808019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f807effd700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8080101940 0x7f8080198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f807e7fc700 1 -- 172.21.15.26:0/3725096013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8080107990 msgr2=0x7f808019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f807e7fc700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8080107990 0x7f808019c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f807e7fc700 1 -- 172.21.15.26:0/3725096013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8080101940 msgr2=0x7f8080198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f807e7fc700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8080101940 0x7f8080198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f807e7fc700 1 -- 172.21.15.26:0/3725096013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808019d000 con 0x7f8080102300 2024-04-21T10:30:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.878+0000 7f807e7fc700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080102300 0x7f8080198580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8074000c00 tx=0x7f80740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.879+0000 7f8067fff700 1 -- 172.21.15.26:0/3725096013 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074027070 con 0x7f8080102300 2024-04-21T10:30:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.879+0000 7f80857db700 1 -- 172.21.15.26:0/3725096013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808019d290 con 0x7f8080102300 2024-04-21T10:30:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.879+0000 7f8067fff700 1 -- 172.21.15.26:0/3725096013 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8074003f20 con 0x7f8080102300 2024-04-21T10:30:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.879+0000 7f8067fff700 1 -- 172.21.15.26:0/3725096013 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074022ca0 con 0x7f8080102300 2024-04-21T10:30:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.879+0000 7f80857db700 1 -- 172.21.15.26:0/3725096013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80801a4b70 con 0x7f8080102300 2024-04-21T10:30:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.880+0000 7f80857db700 1 -- 172.21.15.26:0/3725096013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8080069730 con 0x7f8080102300 2024-04-21T10:30:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.881+0000 7f8067fff700 1 -- 172.21.15.26:0/3725096013 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f807402a070 con 0x7f8080102300 2024-04-21T10:30:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.881+0000 7f8067fff700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f80680a3cd0 0x7f80680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.882+0000 7f807effd700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f80680a3cd0 0x7f80680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.882+0000 7f8067fff700 1 -- 172.21.15.26:0/3725096013 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f80740d1900 con 0x7f8080102300 2024-04-21T10:30:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.882+0000 7f807effd700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f80680a3cd0 0x7f80680a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8070005950 tx=0x7f807000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:01.885+0000 7f8067fff700 1 -- 172.21.15.26:0/3725096013 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8074031070 con 0x7f8080102300 2024-04-21T10:30:02.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.097+0000 7f80857db700 1 -- 172.21.15.26:0/3725096013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f808002df80 con 0x7f8080102300 2024-04-21T10:30:02.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.097+0000 7f8067fff700 1 -- 172.21.15.26:0/3725096013 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8074013070 con 0x7f8080102300 2024-04-21T10:30:02.097 INFO:teuthology.orchestra.run.smithi026.stdout:296352743459 2024-04-21T10:30:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.100+0000 7f80857db700 1 -- 172.21.15.26:0/3725096013 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f80680a3cd0 msgr2=0x7f80680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.100+0000 7f80857db700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f80680a3cd0 0x7f80680a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8070005950 tx=0x7f807000b410 comp rx=0 tx=0).stop 2024-04-21T10:30:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.100+0000 7f80857db700 1 -- 172.21.15.26:0/3725096013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080102300 msgr2=0x7f8080198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.100+0000 7f80857db700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080102300 0x7f8080198580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8074000c00 tx=0x7f80740047b0 comp rx=0 tx=0).stop 2024-04-21T10:30:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.100+0000 7f80857db700 1 -- 172.21.15.26:0/3725096013 shutdown_connections 2024-04-21T10:30:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.101+0000 7f80857db700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f80680a3cd0 0x7f80680a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.101+0000 7f80857db700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8080101940 0x7f8080198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:02.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.101+0000 7f80857db700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8080107990 0x7f808019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:02.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.101+0000 7f80857db700 1 --2- 172.21.15.26:0/3725096013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080102300 0x7f8080198580 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:02.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.101+0000 7f80857db700 1 -- 172.21.15.26:0/3725096013 >> 172.21.15.26:0/3725096013 conn(0x7f80800788f0 msgr2=0x7f80800ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:02.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.101+0000 7f80857db700 1 -- 172.21.15.26:0/3725096013 shutdown_connections 2024-04-21T10:30:02.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:02.101+0000 7f80857db700 1 -- 172.21.15.26:0/3725096013 wait complete. 2024-04-21T10:30:02.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:02 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3725096013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T10:30:02.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-21T10:30:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]: dispatch 2024-04-21T10:30:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-21T10:30:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-21T10:30:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-21T10:30:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-04-21T10:30:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-21T10:30:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-21T10:30:02.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:02 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]: dispatch 2024-04-21T10:30:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:02 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3725096013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T10:30:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-21T10:30:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]: dispatch 2024-04-21T10:30:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-21T10:30:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-21T10:30:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-21T10:30:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-04-21T10:30:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-21T10:30:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-21T10:30:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:02 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]: dispatch 2024-04-21T10:30:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:02 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3725096013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T10:30:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-21T10:30:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]: dispatch 2024-04-21T10:30:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-21T10:30:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-21T10:30:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-21T10:30:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-04-21T10:30:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-21T10:30:02.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-21T10:30:02.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:02 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]: dispatch 2024-04-21T10:30:03.081 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743455 got 296352743459 for osd.9 2024-04-21T10:30:03.081 DEBUG:teuthology.parallel:result is None 2024-04-21T10:30:03.304 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:30:03.304 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-21T10:30:03.304 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]': finished 2024-04-21T10:30:03.304 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-21T10:30:03.305 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-21T10:30:03.305 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-21T10:30:03.622 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-04-21T10:30:03.622 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-21T10:30:03.622 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-21T10:30:03.622 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-04-21T10:30:03.622 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T10:30:03.622 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:03.622 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:03.623 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:03.623 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:03 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:30:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-21T10:30:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]': finished 2024-04-21T10:30:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-21T10:30:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-21T10:30:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-21T10:30:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-04-21T10:30:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-21T10:30:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-21T10:30:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-04-21T10:30:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T10:30:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:03 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:03.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.623+0000 7fe48f33c700 1 -- 172.21.15.26:0/3633051640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe488102300 msgr2=0x7fe48810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:03.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.623+0000 7fe48f33c700 1 --2- 172.21.15.26:0/3633051640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe488102300 0x7fe48810e7f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe48000b0a0 tx=0x7fe48000b3b0 comp rx=0 tx=0).stop 2024-04-21T10:30:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.623+0000 7fe48f33c700 1 -- 172.21.15.26:0/3633051640 shutdown_connections 2024-04-21T10:30:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.623+0000 7fe48f33c700 1 --2- 172.21.15.26:0/3633051640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe488107990 0x7fe488107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.623+0000 7fe48f33c700 1 --2- 172.21.15.26:0/3633051640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe488102300 0x7fe48810e7f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.623+0000 7fe48f33c700 1 --2- 172.21.15.26:0/3633051640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe488101940 0x7fe488101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.623+0000 7fe48f33c700 1 -- 172.21.15.26:0/3633051640 >> 172.21.15.26:0/3633051640 conn(0x7fe4880788f0 msgr2=0x7fe488078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.623+0000 7fe48f33c700 1 -- 172.21.15.26:0/3633051640 shutdown_connections 2024-04-21T10:30:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.623+0000 7fe48f33c700 1 -- 172.21.15.26:0/3633051640 wait complete. 2024-04-21T10:30:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48f33c700 1 Processor -- start 2024-04-21T10:30:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48f33c700 1 -- start start 2024-04-21T10:30:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48f33c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe488101940 0x7fe4881034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48f33c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe488102300 0x7fe488103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48f33c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe488107990 0x7fe488103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48f33c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe488111b70 con 0x7fe488101940 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48f33c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881119f0 con 0x7fe488102300 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48f33c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe488111cf0 con 0x7fe488107990 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48d0d8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe488101940 0x7fe4881034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48d0d8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe488101940 0x7fe4881034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43728/0 (socket says 172.21.15.26:43728) 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48d0d8700 1 -- 172.21.15.26:0/1940699689 learned_addr learned my addr 172.21.15.26:0/1940699689 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48c8d7700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe488102300 0x7fe488103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.624+0000 7fe48d8d9700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe488107990 0x7fe488103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.625+0000 7fe48d0d8700 1 -- 172.21.15.26:0/1940699689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe488107990 msgr2=0x7fe488103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.625+0000 7fe48d0d8700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe488107990 0x7fe488103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.625+0000 7fe48d0d8700 1 -- 172.21.15.26:0/1940699689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe488102300 msgr2=0x7fe488103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.625+0000 7fe48d0d8700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe488102300 0x7fe488103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.625+0000 7fe48d0d8700 1 -- 172.21.15.26:0/1940699689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4881a7640 con 0x7fe488101940 2024-04-21T10:30:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.625+0000 7fe48d0d8700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe488101940 0x7fe4881034f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe47800b530 tx=0x7fe47800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.625+0000 7fe47e7fc700 1 -- 172.21.15.26:0/1940699689 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478015020 con 0x7fe488101940 2024-04-21T10:30:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.625+0000 7fe48f33c700 1 -- 172.21.15.26:0/1940699689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4881a7930 con 0x7fe488101940 2024-04-21T10:30:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.625+0000 7fe47e7fc700 1 -- 172.21.15.26:0/1940699689 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe478011ce0 con 0x7fe488101940 2024-04-21T10:30:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.625+0000 7fe47e7fc700 1 -- 172.21.15.26:0/1940699689 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478010dd0 con 0x7fe488101940 2024-04-21T10:30:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.625+0000 7fe48f33c700 1 -- 172.21.15.26:0/1940699689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4881a7ec0 con 0x7fe488101940 2024-04-21T10:30:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.626+0000 7fe48f33c700 1 -- 172.21.15.26:0/1940699689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe488069730 con 0x7fe488101940 2024-04-21T10:30:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.627+0000 7fe47e7fc700 1 -- 172.21.15.26:0/1940699689 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe478018020 con 0x7fe488101940 2024-04-21T10:30:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.628+0000 7fe47e7fc700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe4740a3cd0 0x7fe4740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.628+0000 7fe48c8d7700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe4740a3cd0 0x7fe4740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.628+0000 7fe47e7fc700 1 -- 172.21.15.26:0/1940699689 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fe4780cfb40 con 0x7fe488101940 2024-04-21T10:30:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.628+0000 7fe48c8d7700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe4740a3cd0 0x7fe4740a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe488079030 tx=0x7fe48400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.632+0000 7fe47e7fc700 1 -- 172.21.15.26:0/1940699689 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe47801f020 con 0x7fe488101940 2024-04-21T10:30:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:30:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-21T10:30:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]': finished 2024-04-21T10:30:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-21T10:30:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-21T10:30:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-21T10:30:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-04-21T10:30:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-21T10:30:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-21T10:30:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-04-21T10:30:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T10:30:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:03 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:03.795 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266727 got 227633266731 for osd.7 2024-04-21T10:30:03.795 DEBUG:teuthology.parallel:result is None 2024-04-21T10:30:03.809 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:03.819 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.868+0000 7fe48f33c700 1 -- 172.21.15.26:0/1940699689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe48802df80 con 0x7fe488101940 2024-04-21T10:30:03.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.868+0000 7fe47e7fc700 1 -- 172.21.15.26:0/1940699689 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe47809c4c0 con 0x7fe488101940 2024-04-21T10:30:03.867 INFO:teuthology.orchestra.run.smithi026.stdout:146028888126 2024-04-21T10:30:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.871+0000 7fe48f33c700 1 -- 172.21.15.26:0/1940699689 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe4740a3cd0 msgr2=0x7fe4740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.871+0000 7fe48f33c700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe4740a3cd0 0x7fe4740a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe488079030 tx=0x7fe48400b3c0 comp rx=0 tx=0).stop 2024-04-21T10:30:03.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.871+0000 7fe48f33c700 1 -- 172.21.15.26:0/1940699689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe488101940 msgr2=0x7fe4881034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.871+0000 7fe48f33c700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe488101940 0x7fe4881034f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe47800b530 tx=0x7fe47800b840 comp rx=0 tx=0).stop 2024-04-21T10:30:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.871+0000 7fe48f33c700 1 -- 172.21.15.26:0/1940699689 shutdown_connections 2024-04-21T10:30:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.871+0000 7fe48f33c700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fe4740a3cd0 0x7fe4740a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.871+0000 7fe48f33c700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe488107990 0x7fe488103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.871+0000 7fe48f33c700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe488101940 0x7fe4881034f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.871+0000 7fe48f33c700 1 --2- 172.21.15.26:0/1940699689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe488102300 0x7fe488103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.871+0000 7fe48f33c700 1 -- 172.21.15.26:0/1940699689 >> 172.21.15.26:0/1940699689 conn(0x7fe4880788f0 msgr2=0x7fe48806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.871+0000 7fe48f33c700 1 -- 172.21.15.26:0/1940699689 shutdown_connections 2024-04-21T10:30:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:03.871+0000 7fe48f33c700 1 -- 172.21.15.26:0/1940699689 wait complete. 2024-04-21T10:30:04.614 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:04 smithi026 ceph-mon[27965]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T10:30:04.615 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:04 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1940699689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T10:30:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.654+0000 7f535620e700 1 -- 172.21.15.26:0/3211357672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f535006b0f0 msgr2=0x7f535010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.654+0000 7f535620e700 1 --2- 172.21.15.26:0/3211357672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f535006b0f0 0x7f535010f900 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f534800b0a0 tx=0x7f534800b3b0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.654+0000 7f535620e700 1 -- 172.21.15.26:0/3211357672 shutdown_connections 2024-04-21T10:30:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.654+0000 7f535620e700 1 --2- 172.21.15.26:0/3211357672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5350108a30 0x7f5350108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.654+0000 7f535620e700 1 --2- 172.21.15.26:0/3211357672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f535006b0f0 0x7f535010f900 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.654+0000 7f535620e700 1 --2- 172.21.15.26:0/3211357672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f535006a750 0x7f535006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.654+0000 7f535620e700 1 -- 172.21.15.26:0/3211357672 >> 172.21.15.26:0/3211357672 conn(0x7f53500788f0 msgr2=0x7f5350078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:04.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.654+0000 7f535620e700 1 -- 172.21.15.26:0/3211357672 shutdown_connections 2024-04-21T10:30:04.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.654+0000 7f535620e700 1 -- 172.21.15.26:0/3211357672 wait complete. 2024-04-21T10:30:04.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.654+0000 7f535620e700 1 Processor -- start 2024-04-21T10:30:04.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.654+0000 7f535620e700 1 -- start start 2024-04-21T10:30:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f535620e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f535006a750 0x7f5350101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f535620e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f535006b0f0 0x7f5350101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:04.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f535620e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5350108a30 0x7f5350102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f535620e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5350112c10 con 0x7f535006b0f0 2024-04-21T10:30:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f535620e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5350112a90 con 0x7f535006a750 2024-04-21T10:30:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f535620e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5350112d90 con 0x7f5350108a30 2024-04-21T10:30:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f534effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f535006b0f0 0x7f5350101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f534effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f535006b0f0 0x7f5350101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44174/0 (socket says 172.21.15.26:44174) 2024-04-21T10:30:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f534effd700 1 -- 172.21.15.26:0/313134438 learned_addr learned my addr 172.21.15.26:0/313134438 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f534f7fe700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f535006a750 0x7f5350101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f534ffff700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5350108a30 0x7f5350102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f534effd700 1 -- 172.21.15.26:0/313134438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5350108a30 msgr2=0x7f5350102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f534effd700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5350108a30 0x7f5350102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f534effd700 1 -- 172.21.15.26:0/313134438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f535006a750 msgr2=0x7f5350101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f534effd700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f535006a750 0x7f5350101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f534effd700 1 -- 172.21.15.26:0/313134438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53501a7690 con 0x7f535006b0f0 2024-04-21T10:30:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f534effd700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f535006b0f0 0x7f5350101f50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f534400d980 tx=0x7f534400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.655+0000 7f534cff9700 1 -- 172.21.15.26:0/313134438 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344017070 con 0x7f535006b0f0 2024-04-21T10:30:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.656+0000 7f534cff9700 1 -- 172.21.15.26:0/313134438 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5344005020 con 0x7f535006b0f0 2024-04-21T10:30:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.656+0000 7f535620e700 1 -- 172.21.15.26:0/313134438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53501a7980 con 0x7f535006b0f0 2024-04-21T10:30:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.656+0000 7f534cff9700 1 -- 172.21.15.26:0/313134438 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344009b50 con 0x7f535006b0f0 2024-04-21T10:30:04.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.656+0000 7f535620e700 1 -- 172.21.15.26:0/313134438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53501a7e70 con 0x7f535006b0f0 2024-04-21T10:30:04.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.657+0000 7f535620e700 1 -- 172.21.15.26:0/313134438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5350069730 con 0x7f535006b0f0 2024-04-21T10:30:04.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.658+0000 7f534cff9700 1 -- 172.21.15.26:0/313134438 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f53440051d0 con 0x7f535006b0f0 2024-04-21T10:30:04.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.659+0000 7f534cff9700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f53380a3cd0 0x7f53380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:04.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.659+0000 7f534f7fe700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f53380a3cd0 0x7f53380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:04.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.659+0000 7f534cff9700 1 -- 172.21.15.26:0/313134438 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f53440d04a0 con 0x7f535006b0f0 2024-04-21T10:30:04.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.659+0000 7f534f7fe700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f53380a3cd0 0x7f53380a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5340005950 tx=0x7f534000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.663+0000 7f534cff9700 1 -- 172.21.15.26:0/313134438 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5344020020 con 0x7f535006b0f0 2024-04-21T10:30:04.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:04 smithi169 ceph-mon[31469]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T10:30:04.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:04 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1940699689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T10:30:04.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:04 smithi190 ceph-mon[31423]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T10:30:04.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:04 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1940699689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T10:30:04.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.812+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/1664762701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbec06afe0 msgr2=0x7fbbec06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.812+0000 7fbbf0ec7700 1 --2- 172.21.15.26:0/1664762701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbec06afe0 0x7fbbec06b440 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009990 tx=0x7fbbe0009ca0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.812+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/1664762701 shutdown_connections 2024-04-21T10:30:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.812+0000 7fbbf0ec7700 1 --2- 172.21.15.26:0/1664762701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbec06b980 0x7fbbec10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.812+0000 7fbbf0ec7700 1 --2- 172.21.15.26:0/1664762701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbec108a20 0x7fbbec108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.812+0000 7fbbf0ec7700 1 --2- 172.21.15.26:0/1664762701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbec06afe0 0x7fbbec06b440 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.812+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/1664762701 >> 172.21.15.26:0/1664762701 conn(0x7fbbec0788b0 msgr2=0x7fbbec078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.812+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/1664762701 shutdown_connections 2024-04-21T10:30:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.812+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/1664762701 wait complete. 2024-04-21T10:30:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.813+0000 7fbbf0ec7700 1 Processor -- start 2024-04-21T10:30:04.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.813+0000 7fbbf0ec7700 1 -- start start 2024-04-21T10:30:04.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.813+0000 7fbbf0ec7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbec06afe0 0x7fbbec101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.813+0000 7fbbf0ec7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbec06b980 0x7fbbec101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.813+0000 7fbbf0ec7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbec108a20 0x7fbbec102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.813+0000 7fbbf0ec7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec112be0 con 0x7fbbec108a20 2024-04-21T10:30:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.813+0000 7fbbf0ec7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec112a60 con 0x7fbbec06afe0 2024-04-21T10:30:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.813+0000 7fbbf0ec7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec112d60 con 0x7fbbec06b980 2024-04-21T10:30:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.813+0000 7fbbead9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbec108a20 0x7fbbec102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.814+0000 7fbbe9d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbec06b980 0x7fbbec101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.814+0000 7fbbea59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbec06afe0 0x7fbbec101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.814+0000 7fbbead9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbec108a20 0x7fbbec102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44196/0 (socket says 172.21.15.26:44196) 2024-04-21T10:30:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.814+0000 7fbbe9d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbec06b980 0x7fbbec101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.26:35400/0 (socket says 172.21.15.26:35400) 2024-04-21T10:30:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.814+0000 7fbbea59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbec06afe0 0x7fbbec101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.26:33914/0 (socket says 172.21.15.26:33914) 2024-04-21T10:30:04.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.814+0000 7fbbead9d700 1 -- 172.21.15.26:0/3245546897 learned_addr learned my addr 172.21.15.26:0/3245546897 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.814+0000 7fbbead9d700 1 -- 172.21.15.26:0/3245546897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbec06b980 msgr2=0x7fbbec101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.814+0000 7fbbead9d700 1 --2- 172.21.15.26:0/3245546897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbec06b980 0x7fbbec101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.814+0000 7fbbead9d700 1 -- 172.21.15.26:0/3245546897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbec06afe0 msgr2=0x7fbbec101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.814+0000 7fbbead9d700 1 --2- 172.21.15.26:0/3245546897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbec06afe0 0x7fbbec101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.814+0000 7fbbead9d700 1 -- 172.21.15.26:0/3245546897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbec1a7550 con 0x7fbbec108a20 2024-04-21T10:30:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.814+0000 7fbbead9d700 1 --2- 172.21.15.26:0/3245546897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbec108a20 0x7fbbec102480 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4007d70 tx=0x7fbbe400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:04.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.817+0000 7fbbdb7fe700 1 -- 172.21.15.26:0/3245546897 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4016020 con 0x7fbbec108a20 2024-04-21T10:30:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.817+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/3245546897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbec1a7840 con 0x7fbbec108a20 2024-04-21T10:30:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.817+0000 7fbbdb7fe700 1 -- 172.21.15.26:0/3245546897 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe400ac90 con 0x7fbbec108a20 2024-04-21T10:30:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.817+0000 7fbbdb7fe700 1 -- 172.21.15.26:0/3245546897 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe40115e0 con 0x7fbbec108a20 2024-04-21T10:30:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.817+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/3245546897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbec1a7d50 con 0x7fbbec108a20 2024-04-21T10:30:04.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.818+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/3245546897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbec069730 con 0x7fbbec108a20 2024-04-21T10:30:04.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.820+0000 7fbbdb7fe700 1 -- 172.21.15.26:0/3245546897 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbbe4019020 con 0x7fbbec108a20 2024-04-21T10:30:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.820+0000 7fbbdb7fe700 1 --2- 172.21.15.26:0/3245546897 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbbd40a3d50 0x7fbbd40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.821+0000 7fbbea59c700 1 --2- 172.21.15.26:0/3245546897 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbbd40a3d50 0x7fbbd40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:04.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.821+0000 7fbbdb7fe700 1 -- 172.21.15.26:0/3245546897 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fbbe40d0410 con 0x7fbbec108a20 2024-04-21T10:30:04.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.821+0000 7fbbea59c700 1 --2- 172.21.15.26:0/3245546897 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbbd40a3d50 0x7fbbd40a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00abd0 tx=0x7fbbdc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:04.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.830+0000 7fbbdb7fe700 1 -- 172.21.15.26:0/3245546897 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbbe4020020 con 0x7fbbec108a20 2024-04-21T10:30:04.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.881+0000 7f535620e700 1 -- 172.21.15.26:0/313134438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f535002df80 con 0x7f535006b0f0 2024-04-21T10:30:04.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.882+0000 7f534cff9700 1 -- 172.21.15.26:0/313134438 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f534409cd10 con 0x7f535006b0f0 2024-04-21T10:30:04.881 INFO:teuthology.orchestra.run.smithi026.stdout:206158430259 2024-04-21T10:30:04.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.884+0000 7f535620e700 1 -- 172.21.15.26:0/313134438 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f53380a3cd0 msgr2=0x7f53380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:04.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.884+0000 7f535620e700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f53380a3cd0 0x7f53380a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5340005950 tx=0x7f534000b410 comp rx=0 tx=0).stop 2024-04-21T10:30:04.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.884+0000 7f535620e700 1 -- 172.21.15.26:0/313134438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f535006b0f0 msgr2=0x7f5350101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:04.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.884+0000 7f535620e700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f535006b0f0 0x7f5350101f50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f534400d980 tx=0x7f534400dc90 comp rx=0 tx=0).stop 2024-04-21T10:30:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.885+0000 7f535620e700 1 -- 172.21.15.26:0/313134438 shutdown_connections 2024-04-21T10:30:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.885+0000 7f535620e700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f53380a3cd0 0x7f53380a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.885+0000 7f535620e700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f535006a750 0x7f5350101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.885+0000 7f535620e700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5350108a30 0x7f5350102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.885+0000 7f535620e700 1 --2- 172.21.15.26:0/313134438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f535006b0f0 0x7f5350101f50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.885+0000 7f535620e700 1 -- 172.21.15.26:0/313134438 >> 172.21.15.26:0/313134438 conn(0x7f53500788f0 msgr2=0x7f53500ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.886+0000 7f535620e700 1 -- 172.21.15.26:0/313134438 shutdown_connections 2024-04-21T10:30:04.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:04.886+0000 7f535620e700 1 -- 172.21.15.26:0/313134438 wait complete. 2024-04-21T10:30:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.053+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/3245546897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fbbec02e140 con 0x7fbbec108a20 2024-04-21T10:30:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.053+0000 7fbbdb7fe700 1 -- 172.21.15.26:0/3245546897 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fbbe409cd30 con 0x7fbbec108a20 2024-04-21T10:30:05.052 INFO:teuthology.orchestra.run.smithi026.stdout:317827579935 2024-04-21T10:30:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.055+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/3245546897 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbbd40a3d50 msgr2=0x7fbbd40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.055+0000 7fbbf0ec7700 1 --2- 172.21.15.26:0/3245546897 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbbd40a3d50 0x7fbbd40a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00abd0 tx=0x7fbbdc00a380 comp rx=0 tx=0).stop 2024-04-21T10:30:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.055+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/3245546897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbec108a20 msgr2=0x7fbbec102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.055+0000 7fbbf0ec7700 1 --2- 172.21.15.26:0/3245546897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbec108a20 0x7fbbec102480 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4007d70 tx=0x7fbbe400a340 comp rx=0 tx=0).stop 2024-04-21T10:30:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.055+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/3245546897 shutdown_connections 2024-04-21T10:30:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.055+0000 7fbbf0ec7700 1 --2- 172.21.15.26:0/3245546897 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbbd40a3d50 0x7fbbd40a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.055+0000 7fbbf0ec7700 1 --2- 172.21.15.26:0/3245546897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbec06afe0 0x7fbbec101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.055+0000 7fbbf0ec7700 1 --2- 172.21.15.26:0/3245546897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbec108a20 0x7fbbec102480 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.055+0000 7fbbf0ec7700 1 --2- 172.21.15.26:0/3245546897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbec06b980 0x7fbbec101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.055+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/3245546897 >> 172.21.15.26:0/3245546897 conn(0x7fbbec0788b0 msgr2=0x7fbbec0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.055+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/3245546897 shutdown_connections 2024-04-21T10:30:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:05.055+0000 7fbbf0ec7700 1 -- 172.21.15.26:0/3245546897 wait complete. 2024-04-21T10:30:05.289 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888120 got 146028888126 for osd.4 2024-04-21T10:30:05.289 DEBUG:teuthology.parallel:result is None 2024-04-21T10:30:05.615 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:05 smithi026 ceph-mon[27965]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-21T10:30:05.617 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:05 smithi026 ceph-mon[27965]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-21T10:30:05.617 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:05 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/313134438' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T10:30:05.617 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:05 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3245546897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T10:30:05.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:05 smithi169 ceph-mon[31469]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-21T10:30:05.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:05 smithi169 ceph-mon[31469]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-21T10:30:05.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:05 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/313134438' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T10:30:05.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:05 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3245546897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T10:30:05.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:05 smithi190 ceph-mon[31423]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-04-21T10:30:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:05 smithi190 ceph-mon[31423]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-21T10:30:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:05 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/313134438' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T10:30:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:05 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3245546897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T10:30:05.948 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579929 got 317827579935 for osd.10 2024-04-21T10:30:05.948 DEBUG:teuthology.parallel:result is None 2024-04-21T10:30:07.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:07 smithi026 ceph-mon[27965]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-21T10:30:07.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:07 smithi169 ceph-mon[31469]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-21T10:30:07.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:07 smithi190 ceph-mon[31423]: pgmap v321: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-21T10:30:07.998 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430254 got 206158430259 for osd.6 2024-04-21T10:30:07.998 DEBUG:teuthology.parallel:result is None 2024-04-21T10:30:08.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.314+0000 7f0bc2c24700 1 -- 172.21.15.26:0/2500949078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc101d20 msgr2=0x7f0bbc1021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.314+0000 7f0bc2c24700 1 --2- 172.21.15.26:0/2500949078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc101d20 0x7f0bbc1021a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009990 tx=0x7f0ba8009ca0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.314+0000 7f0bc2c24700 1 -- 172.21.15.26:0/2500949078 shutdown_connections 2024-04-21T10:30:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.314+0000 7f0bc2c24700 1 --2- 172.21.15.26:0/2500949078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bbc106990 0x7f0bbc106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.314+0000 7f0bc2c24700 1 --2- 172.21.15.26:0/2500949078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bbc1026e0 0x7f0bbc10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.314+0000 7f0bc2c24700 1 --2- 172.21.15.26:0/2500949078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc101d20 0x7f0bbc1021a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.314+0000 7f0bc2c24700 1 -- 172.21.15.26:0/2500949078 >> 172.21.15.26:0/2500949078 conn(0x7f0bbc0788f0 msgr2=0x7f0bbc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.314+0000 7f0bc2c24700 1 -- 172.21.15.26:0/2500949078 shutdown_connections 2024-04-21T10:30:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.314+0000 7f0bc2c24700 1 -- 172.21.15.26:0/2500949078 wait complete. 2024-04-21T10:30:08.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.314+0000 7f0bc2c24700 1 Processor -- start 2024-04-21T10:30:08.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.314+0000 7f0bc2c24700 1 -- start start 2024-04-21T10:30:08.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc2c24700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc101d20 0x7f0bbc19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc2c24700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bbc1026e0 0x7f0bbc19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc2c24700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bbc106990 0x7f0bbc19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc2c24700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc111b30 con 0x7f0bbc101d20 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc2c24700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1119b0 con 0x7f0bbc106990 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc2c24700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc111cb0 con 0x7f0bbc1026e0 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc09c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc101d20 0x7f0bbc19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc09c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc101d20 0x7f0bbc19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44202/0 (socket says 172.21.15.26:44202) 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc09c0700 1 -- 172.21.15.26:0/3406604926 learned_addr learned my addr 172.21.15.26:0/3406604926 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bbbfff700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bbc1026e0 0x7f0bbc19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc11c1700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bbc106990 0x7f0bbc19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc09c0700 1 -- 172.21.15.26:0/3406604926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bbc1026e0 msgr2=0x7f0bbc19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc09c0700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bbc1026e0 0x7f0bbc19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc09c0700 1 -- 172.21.15.26:0/3406604926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bbc106990 msgr2=0x7f0bbc19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc09c0700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bbc106990 0x7f0bbc19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc09c0700 1 -- 172.21.15.26:0/3406604926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc1a0290 con 0x7f0bbc101d20 2024-04-21T10:30:08.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc11c1700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bbc106990 0x7f0bbc19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:08.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.315+0000 7f0bc09c0700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc101d20 0x7f0bbc19b280 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000c9b0 tx=0x7f0bb000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:08.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.316+0000 7f0bb9ffb700 1 -- 172.21.15.26:0/3406604926 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0007b00 con 0x7f0bbc101d20 2024-04-21T10:30:08.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.316+0000 7f0bc2c24700 1 -- 172.21.15.26:0/3406604926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc1a0580 con 0x7f0bbc101d20 2024-04-21T10:30:08.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.316+0000 7f0bb9ffb700 1 -- 172.21.15.26:0/3406604926 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb0007ca0 con 0x7f0bbc101d20 2024-04-21T10:30:08.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.316+0000 7f0bb9ffb700 1 -- 172.21.15.26:0/3406604926 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0018660 con 0x7f0bbc101d20 2024-04-21T10:30:08.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.316+0000 7f0bc2c24700 1 -- 172.21.15.26:0/3406604926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc1a7e00 con 0x7f0bbc101d20 2024-04-21T10:30:08.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.317+0000 7f0bc2c24700 1 -- 172.21.15.26:0/3406604926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bbc103080 con 0x7f0bbc101d20 2024-04-21T10:30:08.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.318+0000 7f0bb9ffb700 1 -- 172.21.15.26:0/3406604926 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0bb000f450 con 0x7f0bbc101d20 2024-04-21T10:30:08.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.318+0000 7f0bb9ffb700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0bac0a3d50 0x7f0bac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:08.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.319+0000 7f0bb9ffb700 1 -- 172.21.15.26:0/3406604926 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f0bb00d0480 con 0x7f0bbc101d20 2024-04-21T10:30:08.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.319+0000 7f0bbbfff700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0bac0a3d50 0x7f0bac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:08.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.319+0000 7f0bbbfff700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0bac0a3d50 0x7f0bac0a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc101b10 tx=0x7f0ba8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:08.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.323+0000 7f0bb9ffb700 1 -- 172.21.15.26:0/3406604926 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0bb0014030 con 0x7f0bbc101d20 2024-04-21T10:30:08.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.523+0000 7fdb2153b700 1 -- 172.21.15.26:0/4009682318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c06b0f0 msgr2=0x7fdb1c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:08.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.523+0000 7fdb2153b700 1 --2- 172.21.15.26:0/4009682318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c06b0f0 0x7fdb1c10f900 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009990 tx=0x7fdb14009ca0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.523+0000 7fdb2153b700 1 -- 172.21.15.26:0/4009682318 shutdown_connections 2024-04-21T10:30:08.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.523+0000 7fdb2153b700 1 --2- 172.21.15.26:0/4009682318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb1c108a30 0x7fdb1c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.523+0000 7fdb2153b700 1 --2- 172.21.15.26:0/4009682318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c06b0f0 0x7fdb1c10f900 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.523+0000 7fdb2153b700 1 --2- 172.21.15.26:0/4009682318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1c06a750 0x7fdb1c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.523+0000 7fdb2153b700 1 -- 172.21.15.26:0/4009682318 >> 172.21.15.26:0/4009682318 conn(0x7fdb1c0788f0 msgr2=0x7fdb1c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:08.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.523+0000 7fdb2153b700 1 -- 172.21.15.26:0/4009682318 shutdown_connections 2024-04-21T10:30:08.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.523+0000 7fdb2153b700 1 -- 172.21.15.26:0/4009682318 wait complete. 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb2153b700 1 Processor -- start 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb2153b700 1 -- start start 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb2153b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb1c06a750 0x7fdb1c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb2153b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c06b0f0 0x7fdb1c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb2153b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1c108a30 0x7fdb1c19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb2153b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c112c10 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb2153b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c112a90 con 0x7fdb1c06a750 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb2153b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c112d90 con 0x7fdb1c108a30 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb1a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c06b0f0 0x7fdb1c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb1a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c06b0f0 0x7fdb1c199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44222/0 (socket says 172.21.15.26:44222) 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb1a7fc700 1 -- 172.21.15.26:0/3752244714 learned_addr learned my addr 172.21.15.26:0/3752244714 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb1affd700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb1c06a750 0x7fdb1c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb1b7fe700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1c108a30 0x7fdb1c19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb1a7fc700 1 -- 172.21.15.26:0/3752244714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1c108a30 msgr2=0x7fdb1c19d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb1a7fc700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1c108a30 0x7fdb1c19d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb1a7fc700 1 -- 172.21.15.26:0/3752244714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb1c06a750 msgr2=0x7fdb1c1990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.524+0000 7fdb1a7fc700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb1c06a750 0x7fdb1c1990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.525+0000 7fdb1a7fc700 1 -- 172.21.15.26:0/3752244714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c19e0f0 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.525+0000 7fdb1b7fe700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1c108a30 0x7fdb1c19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.525+0000 7fdb1a7fc700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c06b0f0 0x7fdb1c199620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400e9c0 tx=0x7fdb0400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.525+0000 7fdb13fff700 1 -- 172.21.15.26:0/3752244714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0400cbd0 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.525+0000 7fdb2153b700 1 -- 172.21.15.26:0/3752244714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c19e3e0 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.525+0000 7fdb13fff700 1 -- 172.21.15.26:0/3752244714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb0400cd70 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.525+0000 7fdb2153b700 1 -- 172.21.15.26:0/3752244714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c1a5c60 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.525+0000 7fdb13fff700 1 -- 172.21.15.26:0/3752244714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04005610 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.526+0000 7fdb2153b700 1 -- 172.21.15.26:0/3752244714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb1c069730 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.527+0000 7fdb13fff700 1 -- 172.21.15.26:0/3752244714 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdb04004750 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.528+0000 7fdb13fff700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fdb080a3cd0 0x7fdb080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:08.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.528+0000 7fdb1affd700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fdb080a3cd0 0x7fdb080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:08.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.528+0000 7fdb13fff700 1 -- 172.21.15.26:0/3752244714 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fdb040d0b30 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.529+0000 7fdb1affd700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fdb080a3cd0 0x7fdb080a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00f540 tx=0x7fdb0c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:08.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.532+0000 7fdb13fff700 1 -- 172.21.15.26:0/3752244714 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb04053ff0 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.538+0000 7f0bc2c24700 1 -- 172.21.15.26:0/3406604926 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0bbc1a8030 con 0x7f0bbc101d20 2024-04-21T10:30:08.537 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:08 smithi026 ceph-mon[27965]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-21T10:30:08.537 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:08 smithi026 ceph-mon[27965]: Cluster is now healthy 2024-04-21T10:30:08.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.539+0000 7f0bb9ffb700 1 -- 172.21.15.26:0/3406604926 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0bb009ccf0 con 0x7f0bbc101d20 2024-04-21T10:30:08.538 INFO:teuthology.orchestra.run.smithi026.stdout:111669149763 2024-04-21T10:30:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.541+0000 7f0bc2c24700 1 -- 172.21.15.26:0/3406604926 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0bac0a3d50 msgr2=0x7f0bac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.541+0000 7f0bc2c24700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0bac0a3d50 0x7f0bac0a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc101b10 tx=0x7f0ba8005fd0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.541+0000 7f0bc2c24700 1 -- 172.21.15.26:0/3406604926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc101d20 msgr2=0x7f0bbc19b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.541+0000 7f0bc2c24700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc101d20 0x7f0bbc19b280 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000c9b0 tx=0x7f0bb000ccc0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.541+0000 7f0bc2c24700 1 -- 172.21.15.26:0/3406604926 shutdown_connections 2024-04-21T10:30:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.541+0000 7f0bc2c24700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0bac0a3d50 0x7f0bac0a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.541+0000 7f0bc2c24700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bbc106990 0x7f0bbc19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.541+0000 7f0bc2c24700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc101d20 0x7f0bbc19b280 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.541+0000 7f0bc2c24700 1 --2- 172.21.15.26:0/3406604926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bbc1026e0 0x7f0bbc19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.541+0000 7f0bc2c24700 1 -- 172.21.15.26:0/3406604926 >> 172.21.15.26:0/3406604926 conn(0x7f0bbc0788f0 msgr2=0x7f0bbc10cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.541+0000 7f0bc2c24700 1 -- 172.21.15.26:0/3406604926 shutdown_connections 2024-04-21T10:30:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.541+0000 7f0bc2c24700 1 -- 172.21.15.26:0/3406604926 wait complete. 2024-04-21T10:30:08.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:08 smithi169 ceph-mon[31469]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-21T10:30:08.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:08 smithi169 ceph-mon[31469]: Cluster is now healthy 2024-04-21T10:30:08.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.754+0000 7fdb2153b700 1 -- 172.21.15.26:0/3752244714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fdb1c02df80 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.755+0000 7fdb13fff700 1 -- 172.21.15.26:0/3752244714 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fdb0409d3a0 con 0x7fdb1c06b0f0 2024-04-21T10:30:08.754 INFO:teuthology.orchestra.run.smithi026.stdout:352187318300 2024-04-21T10:30:08.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.757+0000 7fdb2153b700 1 -- 172.21.15.26:0/3752244714 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fdb080a3cd0 msgr2=0x7fdb080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.757+0000 7fdb2153b700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fdb080a3cd0 0x7fdb080a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00f540 tx=0x7fdb0c00b040 comp rx=0 tx=0).stop 2024-04-21T10:30:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.757+0000 7fdb2153b700 1 -- 172.21.15.26:0/3752244714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c06b0f0 msgr2=0x7fdb1c199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.757+0000 7fdb2153b700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c06b0f0 0x7fdb1c199620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400e9c0 tx=0x7fdb0400ecd0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.757+0000 7fdb2153b700 1 -- 172.21.15.26:0/3752244714 shutdown_connections 2024-04-21T10:30:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.757+0000 7fdb2153b700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fdb080a3cd0 0x7fdb080a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.757+0000 7fdb2153b700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb1c06a750 0x7fdb1c1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.757+0000 7fdb2153b700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1c108a30 0x7fdb1c19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.757+0000 7fdb2153b700 1 --2- 172.21.15.26:0/3752244714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb1c06b0f0 0x7fdb1c199620 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.757+0000 7fdb2153b700 1 -- 172.21.15.26:0/3752244714 >> 172.21.15.26:0/3752244714 conn(0x7fdb1c0788f0 msgr2=0x7fdb1c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.757+0000 7fdb2153b700 1 -- 172.21.15.26:0/3752244714 shutdown_connections 2024-04-21T10:30:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:08.757+0000 7fdb2153b700 1 -- 172.21.15.26:0/3752244714 wait complete. 2024-04-21T10:30:08.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:08 smithi190 ceph-mon[31423]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-21T10:30:08.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:08 smithi190 ceph-mon[31423]: Cluster is now healthy 2024-04-21T10:30:09.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:09 smithi026 ceph-mon[27965]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-04-21T10:30:09.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:09 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3406604926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T10:30:09.573 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:09 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3752244714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T10:30:09.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:09 smithi169 ceph-mon[31469]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-04-21T10:30:09.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:09 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3406604926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T10:30:09.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:09 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3752244714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T10:30:09.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:09 smithi190 ceph-mon[31423]: pgmap v322: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-04-21T10:30:09.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:09 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3406604926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T10:30:09.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:09 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3752244714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T10:30:10.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.360+0000 7f27b5a9b700 1 -- 172.21.15.26:0/3807379497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27b0103480 msgr2=0x7f27b010f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:10.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.360+0000 7f27b5a9b700 1 --2- 172.21.15.26:0/3807379497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27b0103480 0x7f27b010f8d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009990 tx=0x7f27a8009ca0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.360+0000 7f27b5a9b700 1 -- 172.21.15.26:0/3807379497 shutdown_connections 2024-04-21T10:30:10.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.360+0000 7f27b5a9b700 1 --2- 172.21.15.26:0/3807379497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0108a70 0x7f27b0108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.360+0000 7f27b5a9b700 1 --2- 172.21.15.26:0/3807379497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27b0103480 0x7f27b010f8d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.360+0000 7f27b5a9b700 1 --2- 172.21.15.26:0/3807379497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b0102ac0 0x7f27b0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.360+0000 7f27b5a9b700 1 -- 172.21.15.26:0/3807379497 >> 172.21.15.26:0/3807379497 conn(0x7f27b0079bb0 msgr2=0x7f27b0079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:10.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.360+0000 7f27b5a9b700 1 -- 172.21.15.26:0/3807379497 shutdown_connections 2024-04-21T10:30:10.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.360+0000 7f27b5a9b700 1 -- 172.21.15.26:0/3807379497 wait complete. 2024-04-21T10:30:10.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.361+0000 7f27b5a9b700 1 Processor -- start 2024-04-21T10:30:10.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.361+0000 7f27b5a9b700 1 -- start start 2024-04-21T10:30:10.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.361+0000 7f27b5a9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0102ac0 0x7f27b0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:10.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.361+0000 7f27b5a9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27b0103480 0x7f27b01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.361+0000 7f27b5a9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b0108a70 0x7f27b019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.361+0000 7f27b5a9b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0112d00 con 0x7f27b0103480 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.361+0000 7f27b5a9b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0112b80 con 0x7f27b0102ac0 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.361+0000 7f27b5a9b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0112e80 con 0x7f27b0108a70 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.361+0000 7f27aeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27b0103480 0x7f27b01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.361+0000 7f27aeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27b0103480 0x7f27b01997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44240/0 (socket says 172.21.15.26:44240) 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.361+0000 7f27aeffd700 1 -- 172.21.15.26:0/2133300220 learned_addr learned my addr 172.21.15.26:0/2133300220 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.361+0000 7f27affff700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b0108a70 0x7f27b019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.362+0000 7f27af7fe700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0102ac0 0x7f27b0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.362+0000 7f27aeffd700 1 -- 172.21.15.26:0/2133300220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b0108a70 msgr2=0x7f27b019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.362+0000 7f27aeffd700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b0108a70 0x7f27b019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.362+0000 7f27aeffd700 1 -- 172.21.15.26:0/2133300220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0102ac0 msgr2=0x7f27b0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.362+0000 7f27aeffd700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0102ac0 0x7f27b0199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.362+0000 7f27aeffd700 1 -- 172.21.15.26:0/2133300220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b019e2a0 con 0x7f27b0103480 2024-04-21T10:30:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.362+0000 7f27aeffd700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27b0103480 0x7f27b01997d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f279800b530 tx=0x7f279800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.362+0000 7f27acff9700 1 -- 172.21.15.26:0/2133300220 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798015020 con 0x7f27b0103480 2024-04-21T10:30:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.362+0000 7f27b5a9b700 1 -- 172.21.15.26:0/2133300220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b019e590 con 0x7f27b0103480 2024-04-21T10:30:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.362+0000 7f27acff9700 1 -- 172.21.15.26:0/2133300220 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2798011ce0 con 0x7f27b0103480 2024-04-21T10:30:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.362+0000 7f27acff9700 1 -- 172.21.15.26:0/2133300220 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798010e40 con 0x7f27b0103480 2024-04-21T10:30:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.362+0000 7f27b5a9b700 1 -- 172.21.15.26:0/2133300220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b01a5e10 con 0x7f27b0103480 2024-04-21T10:30:10.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.363+0000 7f27b5a9b700 1 -- 172.21.15.26:0/2133300220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b0069730 con 0x7f27b0103480 2024-04-21T10:30:10.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.367+0000 7f27acff9700 1 -- 172.21.15.26:0/2133300220 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2798018020 con 0x7f27b0103480 2024-04-21T10:30:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.368+0000 7f27acff9700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f279c0a3cd0 0x7f279c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.368+0000 7f27af7fe700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f279c0a3cd0 0x7f279c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.368+0000 7f27acff9700 1 -- 172.21.15.26:0/2133300220 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f27980d02b0 con 0x7f27b0103480 2024-04-21T10:30:10.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.368+0000 7f27af7fe700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f279c0a3cd0 0x7f279c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009f20 tx=0x7f27a00094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:10.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.371+0000 7f27acff9700 1 -- 172.21.15.26:0/2133300220 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f279801f020 con 0x7f27b0103480 2024-04-21T10:30:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.578+0000 7fcd6f747700 1 -- 172.21.15.26:0/3389229997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd68108a20 msgr2=0x7fcd68108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.578+0000 7fcd6f747700 1 --2- 172.21.15.26:0/3389229997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd68108a20 0x7fcd68108e00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009990 tx=0x7fcd64009ca0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.578+0000 7fcd6f747700 1 -- 172.21.15.26:0/3389229997 shutdown_connections 2024-04-21T10:30:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.578+0000 7fcd6f747700 1 --2- 172.21.15.26:0/3389229997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6806b980 0x7fcd6810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.578+0000 7fcd6f747700 1 --2- 172.21.15.26:0/3389229997 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd68108a20 0x7fcd68108e00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.578+0000 7fcd6f747700 1 --2- 172.21.15.26:0/3389229997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd6806afe0 0x7fcd6806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.578+0000 7fcd6f747700 1 -- 172.21.15.26:0/3389229997 >> 172.21.15.26:0/3389229997 conn(0x7fcd680788b0 msgr2=0x7fcd68078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.578+0000 7fcd6f747700 1 -- 172.21.15.26:0/3389229997 shutdown_connections 2024-04-21T10:30:10.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.578+0000 7fcd6f747700 1 -- 172.21.15.26:0/3389229997 wait complete. 2024-04-21T10:30:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.578+0000 7fcd6f747700 1 Processor -- start 2024-04-21T10:30:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.578+0000 7fcd6f747700 1 -- start start 2024-04-21T10:30:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6f747700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd6806afe0 0x7fcd68199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6f747700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6806b980 0x7fcd681995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6f747700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd68108a20 0x7fcd6819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:10.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6f747700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcd68112bd0 con 0x7fcd68108a20 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6f747700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd68112a50 con 0x7fcd6806b980 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6f747700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcd68112d50 con 0x7fcd6806afe0 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6dce4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd68108a20 0x7fcd6819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6dce4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd68108a20 0x7fcd6819d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44254/0 (socket says 172.21.15.26:44254) 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6dce4700 1 -- 172.21.15.26:0/997798257 learned_addr learned my addr 172.21.15.26:0/997798257 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6d4e3700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd6806afe0 0x7fcd68199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6cce2700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6806b980 0x7fcd681995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6dce4700 1 -- 172.21.15.26:0/997798257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd6806afe0 msgr2=0x7fcd68199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6dce4700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd6806afe0 0x7fcd68199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6dce4700 1 -- 172.21.15.26:0/997798257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6806b980 msgr2=0x7fcd681995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6dce4700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6806b980 0x7fcd681995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6dce4700 1 -- 172.21.15.26:0/997798257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6819e090 con 0x7fcd68108a20 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6cce2700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6806b980 0x7fcd681995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.579+0000 7fcd6dce4700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd68108a20 0x7fcd6819d950 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcd60007d70 tx=0x7fcd6000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.580+0000 7fcd5e7fc700 1 -- 172.21.15.26:0/997798257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60016020 con 0x7fcd68108a20 2024-04-21T10:30:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.580+0000 7fcd6f747700 1 -- 172.21.15.26:0/997798257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6819e380 con 0x7fcd68108a20 2024-04-21T10:30:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.580+0000 7fcd5e7fc700 1 -- 172.21.15.26:0/997798257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6000ad40 con 0x7fcd68108a20 2024-04-21T10:30:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.580+0000 7fcd5e7fc700 1 -- 172.21.15.26:0/997798257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd600115e0 con 0x7fcd68108a20 2024-04-21T10:30:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.580+0000 7fcd6f747700 1 -- 172.21.15.26:0/997798257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd681a5c00 con 0x7fcd68108a20 2024-04-21T10:30:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.581+0000 7f27b5a9b700 1 -- 172.21.15.26:0/2133300220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f27b002df80 con 0x7f27b0103480 2024-04-21T10:30:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.581+0000 7fcd6f747700 1 -- 172.21.15.26:0/997798257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd68069730 con 0x7fcd68108a20 2024-04-21T10:30:10.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.581+0000 7f27acff9700 1 -- 172.21.15.26:0/2133300220 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f279809cc30 con 0x7f27b0103480 2024-04-21T10:30:10.583 INFO:teuthology.orchestra.run.smithi026.stdout:34359738449 2024-04-21T10:30:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.582+0000 7fcd5e7fc700 1 -- 172.21.15.26:0/997798257 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcd60019020 con 0x7fcd68108a20 2024-04-21T10:30:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.583+0000 7fcd5e7fc700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcd540a3cd0 0x7fcd540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.583+0000 7fcd6d4e3700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcd540a3cd0 0x7fcd540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.583+0000 7fcd5e7fc700 1 -- 172.21.15.26:0/997798257 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fcd600d07d0 con 0x7fcd68108a20 2024-04-21T10:30:10.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7f27b5a9b700 1 -- 172.21.15.26:0/2133300220 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f279c0a3cd0 msgr2=0x7f279c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7f27b5a9b700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f279c0a3cd0 0x7f279c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009f20 tx=0x7f27a00094f0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7f27b5a9b700 1 -- 172.21.15.26:0/2133300220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27b0103480 msgr2=0x7f27b01997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7f27b5a9b700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27b0103480 0x7f27b01997d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f279800b530 tx=0x7f279800b840 comp rx=0 tx=0).stop 2024-04-21T10:30:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7f27b5a9b700 1 -- 172.21.15.26:0/2133300220 shutdown_connections 2024-04-21T10:30:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7f27b5a9b700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f279c0a3cd0 0x7f279c0a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7f27b5a9b700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27b0102ac0 0x7f27b0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7f27b5a9b700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b0108a70 0x7f27b019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7f27b5a9b700 1 --2- 172.21.15.26:0/2133300220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27b0103480 0x7f27b01997d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7f27b5a9b700 1 -- 172.21.15.26:0/2133300220 >> 172.21.15.26:0/2133300220 conn(0x7f27b0079bb0 msgr2=0x7f27b01017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7f27b5a9b700 1 -- 172.21.15.26:0/2133300220 shutdown_connections 2024-04-21T10:30:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7fcd6d4e3700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcd540a3cd0 0x7fcd540a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400b5c0 tx=0x7fcd64005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.584+0000 7f27b5a9b700 1 -- 172.21.15.26:0/2133300220 wait complete. 2024-04-21T10:30:10.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.588+0000 7fcd5e7fc700 1 -- 172.21.15.26:0/997798257 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd60020020 con 0x7fcd68108a20 2024-04-21T10:30:10.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.804+0000 7fcd6f747700 1 -- 172.21.15.26:0/997798257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcd6802df80 con 0x7fcd68108a20 2024-04-21T10:30:10.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.805+0000 7fcd5e7fc700 1 -- 172.21.15.26:0/997798257 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcd60022300 con 0x7fcd68108a20 2024-04-21T10:30:10.805 INFO:teuthology.orchestra.run.smithi026.stdout:171798691898 2024-04-21T10:30:10.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.807+0000 7fcd6f747700 1 -- 172.21.15.26:0/997798257 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcd540a3cd0 msgr2=0x7fcd540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:10.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.807+0000 7fcd6f747700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcd540a3cd0 0x7fcd540a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400b5c0 tx=0x7fcd64005960 comp rx=0 tx=0).stop 2024-04-21T10:30:10.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.807+0000 7fcd6f747700 1 -- 172.21.15.26:0/997798257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd68108a20 msgr2=0x7fcd6819d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:10.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.807+0000 7fcd6f747700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd68108a20 0x7fcd6819d950 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcd60007d70 tx=0x7fcd6000a340 comp rx=0 tx=0).stop 2024-04-21T10:30:10.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.808+0000 7fcd6f747700 1 -- 172.21.15.26:0/997798257 shutdown_connections 2024-04-21T10:30:10.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.808+0000 7fcd6f747700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcd540a3cd0 0x7fcd540a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.808+0000 7fcd6f747700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd68108a20 0x7fcd6819d950 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.808+0000 7fcd6f747700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd6806afe0 0x7fcd68199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.808+0000 7fcd6f747700 1 --2- 172.21.15.26:0/997798257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6806b980 0x7fcd681995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:10.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.808+0000 7fcd6f747700 1 -- 172.21.15.26:0/997798257 >> 172.21.15.26:0/997798257 conn(0x7fcd680788b0 msgr2=0x7fcd680ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:10.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.808+0000 7fcd6f747700 1 -- 172.21.15.26:0/997798257 shutdown_connections 2024-04-21T10:30:10.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:10.808+0000 7fcd6f747700 1 -- 172.21.15.26:0/997798257 wait complete. 2024-04-21T10:30:11.203 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318295 got 352187318300 for osd.11 2024-04-21T10:30:11.204 DEBUG:teuthology.parallel:result is None 2024-04-21T10:30:11.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:11 smithi026 ceph-mon[27965]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 13 B/s, 1 objects/s recovering 2024-04-21T10:30:11.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:11 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2133300220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T10:30:11.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:11 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/997798257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T10:30:11.592 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149757 got 111669149763 for osd.3 2024-04-21T10:30:11.592 DEBUG:teuthology.parallel:result is None 2024-04-21T10:30:11.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:11 smithi169 ceph-mon[31469]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 13 B/s, 1 objects/s recovering 2024-04-21T10:30:11.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:11 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2133300220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T10:30:11.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:11 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/997798257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T10:30:11.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:11 smithi190 ceph-mon[31423]: pgmap v323: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s; 13 B/s, 1 objects/s recovering 2024-04-21T10:30:11.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:11 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2133300220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T10:30:11.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:11 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/997798257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T10:30:12.895 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691894 got 171798691898 for osd.5 2024-04-21T10:30:12.895 DEBUG:teuthology.parallel:result is None 2024-04-21T10:30:13.010 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738449 for osd.0 2024-04-21T10:30:13.010 DEBUG:teuthology.parallel:result is None 2024-04-21T10:30:13.011 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-21T10:30:13.011 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-21T10:30:13.373 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:13.641 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:13 smithi026 ceph-mon[27965]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 11 B/s, 1 objects/s recovering 2024-04-21T10:30:13.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:13 smithi169 ceph-mon[31469]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 11 B/s, 1 objects/s recovering 2024-04-21T10:30:13.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:13 smithi190 ceph-mon[31423]: pgmap v324: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 11 B/s, 1 objects/s recovering 2024-04-21T10:30:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.119+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1826661440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e441026e0 msgr2=0x7f0e4410e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.119+0000 7f0e48fbb700 1 --2- 172.21.15.26:0/1826661440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e441026e0 0x7f0e4410e770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00b0a0 tx=0x7f0e3c00b3b0 comp rx=0 tx=0).stop 2024-04-21T10:30:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.119+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1826661440 shutdown_connections 2024-04-21T10:30:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.119+0000 7f0e48fbb700 1 --2- 172.21.15.26:0/1826661440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e441026e0 0x7f0e4410e770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.119+0000 7f0e48fbb700 1 --2- 172.21.15.26:0/1826661440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e44106990 0x7f0e44106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.119+0000 7f0e48fbb700 1 --2- 172.21.15.26:0/1826661440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e44101d20 0x7f0e441021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.119+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1826661440 >> 172.21.15.26:0/1826661440 conn(0x7f0e440788f0 msgr2=0x7f0e44078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.119+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1826661440 shutdown_connections 2024-04-21T10:30:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.119+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1826661440 wait complete. 2024-04-21T10:30:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.120+0000 7f0e48fbb700 1 Processor -- start 2024-04-21T10:30:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.120+0000 7f0e48fbb700 1 -- start start 2024-04-21T10:30:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.120+0000 7f0e48fbb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e44101d20 0x7f0e44198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.120+0000 7f0e48fbb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e441026e0 0x7f0e44199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.120+0000 7f0e48fbb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e44106990 0x7f0e4419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.120+0000 7f0e48fbb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e44111b40 con 0x7f0e441026e0 2024-04-21T10:30:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.120+0000 7f0e48fbb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e441119c0 con 0x7f0e44101d20 2024-04-21T10:30:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.120+0000 7f0e48fbb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0e44111cc0 con 0x7f0e44106990 2024-04-21T10:30:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.120+0000 7f0e41d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e441026e0 0x7f0e44199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.120+0000 7f0e41d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e441026e0 0x7f0e44199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47774/0 (socket says 172.21.15.26:47774) 2024-04-21T10:30:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.120+0000 7f0e41d9b700 1 -- 172.21.15.26:0/1338324919 learned_addr learned my addr 172.21.15.26:0/1338324919 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.120+0000 7f0e4259c700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e44101d20 0x7f0e44198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.121+0000 7f0e42d9d700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e44106990 0x7f0e4419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.121+0000 7f0e41d9b700 1 -- 172.21.15.26:0/1338324919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e44106990 msgr2=0x7f0e4419d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.121+0000 7f0e41d9b700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e44106990 0x7f0e4419d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.121+0000 7f0e41d9b700 1 -- 172.21.15.26:0/1338324919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e44101d20 msgr2=0x7f0e44198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.121+0000 7f0e41d9b700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e44101d20 0x7f0e44198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.121+0000 7f0e41d9b700 1 -- 172.21.15.26:0/1338324919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4419e000 con 0x7f0e441026e0 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.121+0000 7f0e41d9b700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e441026e0 0x7f0e44199530 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400c9b0 tx=0x7f0e3400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.121+0000 7f0e3b7fe700 1 -- 172.21.15.26:0/1338324919 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34007b00 con 0x7f0e441026e0 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.121+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1338324919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4419e2f0 con 0x7f0e441026e0 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.121+0000 7f0e3b7fe700 1 -- 172.21.15.26:0/1338324919 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e34007ca0 con 0x7f0e441026e0 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.121+0000 7f0e3b7fe700 1 -- 172.21.15.26:0/1338324919 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34018660 con 0x7f0e441026e0 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.121+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1338324919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e441a5bf0 con 0x7f0e441026e0 2024-04-21T10:30:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.122+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1338324919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e44103080 con 0x7f0e441026e0 2024-04-21T10:30:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.123+0000 7f0e3b7fe700 1 -- 172.21.15.26:0/1338324919 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0e3400f450 con 0x7f0e441026e0 2024-04-21T10:30:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.124+0000 7f0e3b7fe700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0e300a3d50 0x7f0e300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.124+0000 7f0e4259c700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0e300a3d50 0x7f0e300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.124+0000 7f0e3b7fe700 1 -- 172.21.15.26:0/1338324919 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f0e340d0500 con 0x7f0e441026e0 2024-04-21T10:30:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.124+0000 7f0e4259c700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0e300a3d50 0x7f0e300a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009a10 tx=0x7f0e2c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:15.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.129+0000 7f0e3b7fe700 1 -- 172.21.15.26:0/1338324919 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e34014030 con 0x7f0e441026e0 2024-04-21T10:30:15.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.342+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1338324919 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e44062e80 con 0x7f0e300a3d50 2024-04-21T10:30:15.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.349+0000 7f0e3b7fe700 1 -- 172.21.15.26:0/1338324919 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258722 (secure 0 0 0) 0x7f0e44062e80 con 0x7f0e300a3d50 2024-04-21T10:30:15.352 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.353+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1338324919 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0e300a3d50 msgr2=0x7f0e300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.353+0000 7f0e48fbb700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0e300a3d50 0x7f0e300a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009a10 tx=0x7f0e2c009190 comp rx=0 tx=0).stop 2024-04-21T10:30:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.353+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1338324919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e441026e0 msgr2=0x7f0e44199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.353+0000 7f0e48fbb700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e441026e0 0x7f0e44199530 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400c9b0 tx=0x7f0e3400ccc0 comp rx=0 tx=0).stop 2024-04-21T10:30:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.353+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1338324919 shutdown_connections 2024-04-21T10:30:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.353+0000 7f0e48fbb700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f0e300a3d50 0x7f0e300a6210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.353+0000 7f0e48fbb700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e44101d20 0x7f0e44198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.353+0000 7f0e48fbb700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e44106990 0x7f0e4419d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.353+0000 7f0e48fbb700 1 --2- 172.21.15.26:0/1338324919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e441026e0 0x7f0e44199530 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.353+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1338324919 >> 172.21.15.26:0/1338324919 conn(0x7f0e440788f0 msgr2=0x7f0e4406b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.354+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1338324919 shutdown_connections 2024-04-21T10:30:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:15.354+0000 7f0e48fbb700 1 -- 172.21.15.26:0/1338324919 wait complete. 2024-04-21T10:30:15.356 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-21T10:30:15.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:15 smithi026 ceph-mon[27965]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-04-21T10:30:15.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:15.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:15 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:15.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:15 smithi169 ceph-mon[31469]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-04-21T10:30:15.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:15.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:15 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:15.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:15 smithi190 ceph-mon[31423]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-04-21T10:30:15.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:15.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:15 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:15.954 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-04-21T10:30:14.195795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654626,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":564,"num_read_kb":400,"num_write":341,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362716,"kb_used_data":13500,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579108,"statfs":{"total":1151940427776,"available":1151569006592,"internally_reserved":0,"allocated":13824000,"data_stored":7523322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-9,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.896823"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303593+0000","last_change":"2024-04-21T10:29:01.940996+0000","last_active":"2024-04-21T10:30:03.303593+0000","last_peered":"2024-04-21T10:30:03.303593+0000","last_clean":"2024-04-21T10:30:03.303593+0000","last_became_active":"2024-04-21T10:29:01.940662+0000","last_became_peered":"2024-04-21T10:29:01.940662+0000","last_unstale":"2024-04-21T10:30:03.303593+0000","last_undegraded":"2024-04-21T10:30:03.303593+0000","last_fullsized":"2024-04-21T10:30:03.303593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:16:01.434895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308135+0000","last_change":"2024-04-21T10:29:01.939238+0000","last_active":"2024-04-21T10:30:03.308135+0000","last_peered":"2024-04-21T10:30:03.308135+0000","last_clean":"2024-04-21T10:30:03.308135+0000","last_became_active":"2024-04-21T10:29:01.938907+0000","last_became_peered":"2024-04-21T10:29:01.938907+0000","last_unstale":"2024-04-21T10:30:03.308135+0000","last_undegraded":"2024-04-21T10:30:03.308135+0000","last_fullsized":"2024-04-21T10:30:03.308135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:09.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307936+0000","last_change":"2024-04-21T10:29:01.932338+0000","last_active":"2024-04-21T10:30:03.307936+0000","last_peered":"2024-04-21T10:30:03.307936+0000","last_clean":"2024-04-21T10:30:03.307936+0000","last_became_active":"2024-04-21T10:29:01.931916+0000","last_became_peered":"2024-04-21T10:29:01.931916+0000","last_unstale":"2024-04-21T10:30:03.307936+0000","last_undegraded":"2024-04-21T10:30:03.307936+0000","last_fullsized":"2024-04-21T10:30:03.307936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:52.859695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.519199+0000","last_change":"2024-04-21T10:29:01.945572+0000","last_active":"2024-04-21T10:30:03.519199+0000","last_peered":"2024-04-21T10:30:03.519199+0000","last_clean":"2024-04-21T10:30:03.519199+0000","last_became_active":"2024-04-21T10:29:01.940114+0000","last_became_peered":"2024-04-21T10:29:01.940114+0000","last_unstale":"2024-04-21T10:30:03.519199+0000","last_undegraded":"2024-04-21T10:30:03.519199+0000","last_fullsized":"2024-04-21T10:30:03.519199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:21.935573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301983+0000","last_change":"2024-04-21T10:29:01.946021+0000","last_active":"2024-04-21T10:30:03.301983+0000","last_peered":"2024-04-21T10:30:03.301983+0000","last_clean":"2024-04-21T10:30:03.301983+0000","last_became_active":"2024-04-21T10:29:01.945637+0000","last_became_peered":"2024-04-21T10:29:01.945637+0000","last_unstale":"2024-04-21T10:30:03.301983+0000","last_undegraded":"2024-04-21T10:30:03.301983+0000","last_fullsized":"2024-04-21T10:30:03.301983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:20.840494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.304092+0000","last_change":"2024-04-21T10:29:01.935402+0000","last_active":"2024-04-21T10:30:03.304092+0000","last_peered":"2024-04-21T10:30:03.304092+0000","last_clean":"2024-04-21T10:30:03.304092+0000","last_became_active":"2024-04-21T10:29:01.934994+0000","last_became_peered":"2024-04-21T10:29:01.934994+0000","last_unstale":"2024-04-21T10:30:03.304092+0000","last_undegraded":"2024-04-21T10:30:03.304092+0000","last_fullsized":"2024-04-21T10:30:03.304092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:18.320530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300878+0000","last_change":"2024-04-21T10:29:01.943185+0000","last_active":"2024-04-21T10:30:03.300878+0000","last_peered":"2024-04-21T10:30:03.300878+0000","last_clean":"2024-04-21T10:30:03.300878+0000","last_became_active":"2024-04-21T10:29:01.942805+0000","last_became_peered":"2024-04-21T10:29:01.942805+0000","last_unstale":"2024-04-21T10:30:03.300878+0000","last_undegraded":"2024-04-21T10:30:03.300878+0000","last_fullsized":"2024-04-21T10:30:03.300878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:06.776472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307431+0000","last_change":"2024-04-21T10:29:01.940995+0000","last_active":"2024-04-21T10:30:03.307431+0000","last_peered":"2024-04-21T10:30:03.307431+0000","last_clean":"2024-04-21T10:30:03.307431+0000","last_became_active":"2024-04-21T10:29:01.939173+0000","last_became_peered":"2024-04-21T10:29:01.939173+0000","last_unstale":"2024-04-21T10:30:03.307431+0000","last_undegraded":"2024-04-21T10:30:03.307431+0000","last_fullsized":"2024-04-21T10:30:03.307431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.740147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.519792+0000","last_change":"2024-04-21T10:29:01.944284+0000","last_active":"2024-04-21T10:30:03.519792+0000","last_peered":"2024-04-21T10:30:03.519792+0000","last_clean":"2024-04-21T10:30:03.519792+0000","last_became_active":"2024-04-21T10:29:01.943965+0000","last_became_peered":"2024-04-21T10:29:01.943965+0000","last_unstale":"2024-04-21T10:30:03.519792+0000","last_undegraded":"2024-04-21T10:30:03.519792+0000","last_fullsized":"2024-04-21T10:30:03.519792+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:08:54.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303863+0000","last_change":"2024-04-21T10:29:01.936250+0000","last_active":"2024-04-21T10:30:03.303863+0000","last_peered":"2024-04-21T10:30:03.303863+0000","last_clean":"2024-04-21T10:30:03.303863+0000","last_became_active":"2024-04-21T10:29:01.935904+0000","last_became_peered":"2024-04-21T10:29:01.935904+0000","last_unstale":"2024-04-21T10:30:03.303863+0000","last_undegraded":"2024-04-21T10:30:03.303863+0000","last_fullsized":"2024-04-21T10:30:03.303863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:35.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301084+0000","last_change":"2024-04-21T10:29:01.946550+0000","last_active":"2024-04-21T10:30:03.301084+0000","last_peered":"2024-04-21T10:30:03.301084+0000","last_clean":"2024-04-21T10:30:03.301084+0000","last_became_active":"2024-04-21T10:29:01.946170+0000","last_became_peered":"2024-04-21T10:29:01.946170+0000","last_unstale":"2024-04-21T10:30:03.301084+0000","last_undegraded":"2024-04-21T10:30:03.301084+0000","last_fullsized":"2024-04-21T10:30:03.301084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:25:47.671821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.519100+0000","last_change":"2024-04-21T10:29:01.946036+0000","last_active":"2024-04-21T10:30:03.519100+0000","last_peered":"2024-04-21T10:30:03.519100+0000","last_clean":"2024-04-21T10:30:03.519100+0000","last_became_active":"2024-04-21T10:29:01.940705+0000","last_became_peered":"2024-04-21T10:29:01.940705+0000","last_unstale":"2024-04-21T10:30:03.519100+0000","last_undegraded":"2024-04-21T10:30:03.519100+0000","last_fullsized":"2024-04-21T10:30:03.519100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:11:33.196059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303710+0000","last_change":"2024-04-21T10:29:01.940438+0000","last_active":"2024-04-21T10:30:03.303710+0000","last_peered":"2024-04-21T10:30:03.303710+0000","last_clean":"2024-04-21T10:30:03.303710+0000","last_became_active":"2024-04-21T10:29:01.940085+0000","last_became_peered":"2024-04-21T10:29:01.940085+0000","last_unstale":"2024-04-21T10:30:03.303710+0000","last_undegraded":"2024-04-21T10:30:03.303710+0000","last_fullsized":"2024-04-21T10:30:03.303710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:22:47.828858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307189+0000","last_change":"2024-04-21T10:29:01.941667+0000","last_active":"2024-04-21T10:30:03.307189+0000","last_peered":"2024-04-21T10:30:03.307189+0000","last_clean":"2024-04-21T10:30:03.307189+0000","last_became_active":"2024-04-21T10:29:01.940745+0000","last_became_peered":"2024-04-21T10:29:01.940745+0000","last_unstale":"2024-04-21T10:30:03.307189+0000","last_undegraded":"2024-04-21T10:30:03.307189+0000","last_fullsized":"2024-04-21T10:30:03.307189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:45:51.825780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.306896+0000","last_change":"2024-04-21T10:29:01.947771+0000","last_active":"2024-04-21T10:30:03.306896+0000","last_peered":"2024-04-21T10:30:03.306896+0000","last_clean":"2024-04-21T10:30:03.306896+0000","last_became_active":"2024-04-21T10:29:01.946818+0000","last_became_peered":"2024-04-21T10:29:01.946818+0000","last_unstale":"2024-04-21T10:30:03.306896+0000","last_undegraded":"2024-04-21T10:30:03.306896+0000","last_fullsized":"2024-04-21T10:30:03.306896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:18:42.159152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518778+0000","last_change":"2024-04-21T10:29:01.942499+0000","last_active":"2024-04-21T10:30:03.518778+0000","last_peered":"2024-04-21T10:30:03.518778+0000","last_clean":"2024-04-21T10:30:03.518778+0000","last_became_active":"2024-04-21T10:29:01.942136+0000","last_became_peered":"2024-04-21T10:29:01.942136+0000","last_unstale":"2024-04-21T10:30:03.518778+0000","last_undegraded":"2024-04-21T10:30:03.518778+0000","last_fullsized":"2024-04-21T10:30:03.518778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:34:11.822230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303603+0000","last_change":"2024-04-21T10:29:01.948822+0000","last_active":"2024-04-21T10:30:03.303603+0000","last_peered":"2024-04-21T10:30:03.303603+0000","last_clean":"2024-04-21T10:30:03.303603+0000","last_became_active":"2024-04-21T10:29:01.948492+0000","last_became_peered":"2024-04-21T10:29:01.948492+0000","last_unstale":"2024-04-21T10:30:03.303603+0000","last_undegraded":"2024-04-21T10:30:03.303603+0000","last_fullsized":"2024-04-21T10:30:03.303603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:06.992585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300138+0000","last_change":"2024-04-21T10:29:01.931265+0000","last_active":"2024-04-21T10:30:03.300138+0000","last_peered":"2024-04-21T10:30:03.300138+0000","last_clean":"2024-04-21T10:30:03.300138+0000","last_became_active":"2024-04-21T10:29:01.930877+0000","last_became_peered":"2024-04-21T10:29:01.930877+0000","last_unstale":"2024-04-21T10:30:03.300138+0000","last_undegraded":"2024-04-21T10:30:03.300138+0000","last_fullsized":"2024-04-21T10:30:03.300138+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:49:00.951226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301987+0000","last_change":"2024-04-21T10:29:01.941718+0000","last_active":"2024-04-21T10:30:03.301987+0000","last_peered":"2024-04-21T10:30:03.301987+0000","last_clean":"2024-04-21T10:30:03.301987+0000","last_became_active":"2024-04-21T10:29:01.941386+0000","last_became_peered":"2024-04-21T10:29:01.941386+0000","last_unstale":"2024-04-21T10:30:03.301987+0000","last_undegraded":"2024-04-21T10:30:03.301987+0000","last_fullsized":"2024-04-21T10:30:03.301987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:51:29.847961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300070+0000","last_change":"2024-04-21T10:29:00.923670+0000","last_active":"2024-04-21T10:30:03.300070+0000","last_peered":"2024-04-21T10:30:03.300070+0000","last_clean":"2024-04-21T10:30:03.300070+0000","last_became_active":"2024-04-21T10:29:00.923100+0000","last_became_peered":"2024-04-21T10:29:00.923100+0000","last_unstale":"2024-04-21T10:30:03.300070+0000","last_undegraded":"2024-04-21T10:30:03.300070+0000","last_fullsized":"2024-04-21T10:30:03.300070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:45.692961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307802+0000","last_change":"2024-04-21T10:29:01.938131+0000","last_active":"2024-04-21T10:30:03.307802+0000","last_peered":"2024-04-21T10:30:03.307802+0000","last_clean":"2024-04-21T10:30:03.307802+0000","last_became_active":"2024-04-21T10:29:01.937752+0000","last_became_peered":"2024-04-21T10:29:01.937752+0000","last_unstale":"2024-04-21T10:30:03.307802+0000","last_undegraded":"2024-04-21T10:30:03.307802+0000","last_fullsized":"2024-04-21T10:30:03.307802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:06:49.383554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307979+0000","last_change":"2024-04-21T10:29:00.929877+0000","last_active":"2024-04-21T10:30:03.307979+0000","last_peered":"2024-04-21T10:30:03.307979+0000","last_clean":"2024-04-21T10:30:03.307979+0000","last_became_active":"2024-04-21T10:29:00.929564+0000","last_became_peered":"2024-04-21T10:29:00.929564+0000","last_unstale":"2024-04-21T10:30:03.307979+0000","last_undegraded":"2024-04-21T10:30:03.307979+0000","last_fullsized":"2024-04-21T10:30:03.307979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:35.617533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308242+0000","last_change":"2024-04-21T10:29:01.942782+0000","last_active":"2024-04-21T10:30:03.308242+0000","last_peered":"2024-04-21T10:30:03.308242+0000","last_clean":"2024-04-21T10:30:03.308242+0000","last_became_active":"2024-04-21T10:29:01.940501+0000","last_became_peered":"2024-04-21T10:29:01.940501+0000","last_unstale":"2024-04-21T10:30:03.308242+0000","last_undegraded":"2024-04-21T10:30:03.308242+0000","last_fullsized":"2024-04-21T10:30:03.308242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:04:03.431016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302544+0000","last_change":"2024-04-21T10:29:00.924395+0000","last_active":"2024-04-21T10:30:03.302544+0000","last_peered":"2024-04-21T10:30:03.302544+0000","last_clean":"2024-04-21T10:30:03.302544+0000","last_became_active":"2024-04-21T10:29:00.924045+0000","last_became_peered":"2024-04-21T10:29:00.924045+0000","last_unstale":"2024-04-21T10:30:03.302544+0000","last_undegraded":"2024-04-21T10:30:03.302544+0000","last_fullsized":"2024-04-21T10:30:03.302544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:08:53.922213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518063+0000","last_change":"2024-04-21T10:29:01.934508+0000","last_active":"2024-04-21T10:30:03.518063+0000","last_peered":"2024-04-21T10:30:03.518063+0000","last_clean":"2024-04-21T10:30:03.518063+0000","last_became_active":"2024-04-21T10:29:01.934051+0000","last_became_peered":"2024-04-21T10:29:01.934051+0000","last_unstale":"2024-04-21T10:30:03.518063+0000","last_undegraded":"2024-04-21T10:30:03.518063+0000","last_fullsized":"2024-04-21T10:30:03.518063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:49:27.686603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302573+0000","last_change":"2024-04-21T10:29:00.925850+0000","last_active":"2024-04-21T10:30:03.302573+0000","last_peered":"2024-04-21T10:30:03.302573+0000","last_clean":"2024-04-21T10:30:03.302573+0000","last_became_active":"2024-04-21T10:29:00.925521+0000","last_became_peered":"2024-04-21T10:29:00.925521+0000","last_unstale":"2024-04-21T10:30:03.302573+0000","last_undegraded":"2024-04-21T10:30:03.302573+0000","last_fullsized":"2024-04-21T10:30:03.302573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:36:55.403413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305716+0000","last_change":"2024-04-21T10:29:01.930046+0000","last_active":"2024-04-21T10:30:03.305716+0000","last_peered":"2024-04-21T10:30:03.305716+0000","last_clean":"2024-04-21T10:30:03.305716+0000","last_became_active":"2024-04-21T10:29:01.929736+0000","last_became_peered":"2024-04-21T10:29:01.929736+0000","last_unstale":"2024-04-21T10:30:03.305716+0000","last_undegraded":"2024-04-21T10:30:03.305716+0000","last_fullsized":"2024-04-21T10:30:03.305716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:11:10.833267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:10.910326+0000","last_change":"2024-04-21T10:29:00.927936+0000","last_active":"2024-04-21T10:30:10.910326+0000","last_peered":"2024-04-21T10:30:10.910326+0000","last_clean":"2024-04-21T10:30:10.910326+0000","last_became_active":"2024-04-21T10:29:00.927352+0000","last_became_peered":"2024-04-21T10:29:00.927352+0000","last_unstale":"2024-04-21T10:30:10.910326+0000","last_undegraded":"2024-04-21T10:30:10.910326+0000","last_fullsized":"2024-04-21T10:30:10.910326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:14:10.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.306538+0000","last_change":"2024-04-21T10:30:03.306538+0000","last_active":"2024-04-21T10:30:03.306538+0000","last_peered":"2024-04-21T10:30:03.306538+0000","last_clean":"2024-04-21T10:30:03.306538+0000","last_became_active":"2024-04-21T10:30:03.306197+0000","last_became_peered":"2024-04-21T10:30:03.306197+0000","last_unstale":"2024-04-21T10:30:03.306538+0000","last_undegraded":"2024-04-21T10:30:03.306538+0000","last_fullsized":"2024-04-21T10:30:03.306538+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:28:25.688821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.522106+0000","last_change":"2024-04-21T10:30:03.522106+0000","last_active":"2024-04-21T10:30:03.522106+0000","last_peered":"2024-04-21T10:30:03.522106+0000","last_clean":"2024-04-21T10:30:03.522106+0000","last_became_active":"2024-04-21T10:30:03.519189+0000","last_became_peered":"2024-04-21T10:30:03.519189+0000","last_unstale":"2024-04-21T10:30:03.522106+0000","last_undegraded":"2024-04-21T10:30:03.522106+0000","last_fullsized":"2024-04-21T10:30:03.522106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:49:39.930818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305558+0000","last_change":"2024-04-21T10:29:01.918627+0000","last_active":"2024-04-21T10:30:03.305558+0000","last_peered":"2024-04-21T10:30:03.305558+0000","last_clean":"2024-04-21T10:30:03.305558+0000","last_became_active":"2024-04-21T10:29:01.917993+0000","last_became_peered":"2024-04-21T10:29:01.917993+0000","last_unstale":"2024-04-21T10:30:03.305558+0000","last_undegraded":"2024-04-21T10:30:03.305558+0000","last_fullsized":"2024-04-21T10:30:03.305558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:58.891918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303712+0000","last_change":"2024-04-21T10:29:00.923613+0000","last_active":"2024-04-21T10:30:03.303712+0000","last_peered":"2024-04-21T10:30:03.303712+0000","last_clean":"2024-04-21T10:30:03.303712+0000","last_became_active":"2024-04-21T10:29:00.923001+0000","last_became_peered":"2024-04-21T10:29:00.923001+0000","last_unstale":"2024-04-21T10:30:03.303712+0000","last_undegraded":"2024-04-21T10:30:03.303712+0000","last_fullsized":"2024-04-21T10:30:03.303712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:44:53.593903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307316+0000","last_change":"2024-04-21T10:29:01.934974+0000","last_active":"2024-04-21T10:30:03.307316+0000","last_peered":"2024-04-21T10:30:03.307316+0000","last_clean":"2024-04-21T10:30:03.307316+0000","last_became_active":"2024-04-21T10:29:01.932443+0000","last_became_peered":"2024-04-21T10:29:01.932443+0000","last_unstale":"2024-04-21T10:30:03.307316+0000","last_undegraded":"2024-04-21T10:30:03.307316+0000","last_fullsized":"2024-04-21T10:30:03.307316+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:24:47.190142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305205+0000","last_change":"2024-04-21T10:29:00.928179+0000","last_active":"2024-04-21T10:30:03.305205+0000","last_peered":"2024-04-21T10:30:03.305205+0000","last_clean":"2024-04-21T10:30:03.305205+0000","last_became_active":"2024-04-21T10:29:00.927588+0000","last_became_peered":"2024-04-21T10:29:00.927588+0000","last_unstale":"2024-04-21T10:30:03.305205+0000","last_undegraded":"2024-04-21T10:30:03.305205+0000","last_fullsized":"2024-04-21T10:30:03.305205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:04:41.081424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.299988+0000","last_change":"2024-04-21T10:29:01.944421+0000","last_active":"2024-04-21T10:30:03.299988+0000","last_peered":"2024-04-21T10:30:03.299988+0000","last_clean":"2024-04-21T10:30:03.299988+0000","last_became_active":"2024-04-21T10:29:01.943572+0000","last_became_peered":"2024-04-21T10:29:01.943572+0000","last_unstale":"2024-04-21T10:30:03.299988+0000","last_undegraded":"2024-04-21T10:30:03.299988+0000","last_fullsized":"2024-04-21T10:30:03.299988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:34:01.690600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:10.909801+0000","last_change":"2024-04-21T10:29:00.924406+0000","last_active":"2024-04-21T10:30:10.909801+0000","last_peered":"2024-04-21T10:30:10.909801+0000","last_clean":"2024-04-21T10:30:10.909801+0000","last_became_active":"2024-04-21T10:29:00.924094+0000","last_became_peered":"2024-04-21T10:29:00.924094+0000","last_unstale":"2024-04-21T10:30:10.909801+0000","last_undegraded":"2024-04-21T10:30:10.909801+0000","last_fullsized":"2024-04-21T10:30:10.909801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:44.673751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518659+0000","last_change":"2024-04-21T10:29:01.933048+0000","last_active":"2024-04-21T10:30:03.518659+0000","last_peered":"2024-04-21T10:30:03.518659+0000","last_clean":"2024-04-21T10:30:03.518659+0000","last_became_active":"2024-04-21T10:29:01.932659+0000","last_became_peered":"2024-04-21T10:29:01.932659+0000","last_unstale":"2024-04-21T10:30:03.518659+0000","last_undegraded":"2024-04-21T10:30:03.518659+0000","last_fullsized":"2024-04-21T10:30:03.518659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:22:13.712112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301009+0000","last_change":"2024-04-21T10:29:00.923020+0000","last_active":"2024-04-21T10:30:03.301009+0000","last_peered":"2024-04-21T10:30:03.301009+0000","last_clean":"2024-04-21T10:30:03.301009+0000","last_became_active":"2024-04-21T10:29:00.922678+0000","last_became_peered":"2024-04-21T10:29:00.922678+0000","last_unstale":"2024-04-21T10:30:03.301009+0000","last_undegraded":"2024-04-21T10:30:03.301009+0000","last_fullsized":"2024-04-21T10:30:03.301009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:13:34.698082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308326+0000","last_change":"2024-04-21T10:29:01.947833+0000","last_active":"2024-04-21T10:30:03.308326+0000","last_peered":"2024-04-21T10:30:03.308326+0000","last_clean":"2024-04-21T10:30:03.308326+0000","last_became_active":"2024-04-21T10:29:01.947351+0000","last_became_peered":"2024-04-21T10:29:01.947351+0000","last_unstale":"2024-04-21T10:30:03.308326+0000","last_undegraded":"2024-04-21T10:30:03.308326+0000","last_fullsized":"2024-04-21T10:30:03.308326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:54.538731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305809+0000","last_change":"2024-04-21T10:29:00.932309+0000","last_active":"2024-04-21T10:30:03.305809+0000","last_peered":"2024-04-21T10:30:03.305809+0000","last_clean":"2024-04-21T10:30:03.305809+0000","last_became_active":"2024-04-21T10:29:00.931935+0000","last_became_peered":"2024-04-21T10:29:00.931935+0000","last_unstale":"2024-04-21T10:30:03.305809+0000","last_undegraded":"2024-04-21T10:30:03.305809+0000","last_fullsized":"2024-04-21T10:30:03.305809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:37:37.604381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518174+0000","last_change":"2024-04-21T10:29:01.946543+0000","last_active":"2024-04-21T10:30:03.518174+0000","last_peered":"2024-04-21T10:30:03.518174+0000","last_clean":"2024-04-21T10:30:03.518174+0000","last_became_active":"2024-04-21T10:29:01.946210+0000","last_became_peered":"2024-04-21T10:29:01.946210+0000","last_unstale":"2024-04-21T10:30:03.518174+0000","last_undegraded":"2024-04-21T10:30:03.518174+0000","last_fullsized":"2024-04-21T10:30:03.518174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:49.368851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'30","reported_seq":192,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:11.922141+0000","last_change":"2024-04-21T10:29:00.924109+0000","last_active":"2024-04-21T10:30:11.922141+0000","last_peered":"2024-04-21T10:30:11.922141+0000","last_clean":"2024-04-21T10:30:11.922141+0000","last_became_active":"2024-04-21T10:29:00.923735+0000","last_became_peered":"2024-04-21T10:29:00.923735+0000","last_unstale":"2024-04-21T10:30:11.922141+0000","last_undegraded":"2024-04-21T10:30:11.922141+0000","last_fullsized":"2024-04-21T10:30:11.922141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:13:54.015774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":143,"num_read_kb":130,"num_write":31,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302268+0000","last_change":"2024-04-21T10:29:01.937211+0000","last_active":"2024-04-21T10:30:03.302268+0000","last_peered":"2024-04-21T10:30:03.302268+0000","last_clean":"2024-04-21T10:30:03.302268+0000","last_became_active":"2024-04-21T10:29:01.936891+0000","last_became_peered":"2024-04-21T10:29:01.936891+0000","last_unstale":"2024-04-21T10:30:03.302268+0000","last_undegraded":"2024-04-21T10:30:03.302268+0000","last_fullsized":"2024-04-21T10:30:03.302268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:26.062236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301541+0000","last_change":"2024-04-21T10:29:00.920626+0000","last_active":"2024-04-21T10:30:03.301541+0000","last_peered":"2024-04-21T10:30:03.301541+0000","last_clean":"2024-04-21T10:30:03.301541+0000","last_became_active":"2024-04-21T10:29:00.920317+0000","last_became_peered":"2024-04-21T10:29:00.920317+0000","last_unstale":"2024-04-21T10:30:03.301541+0000","last_undegraded":"2024-04-21T10:30:03.301541+0000","last_fullsized":"2024-04-21T10:30:03.301541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:41:18.373403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303859+0000","last_change":"2024-04-21T10:29:01.942494+0000","last_active":"2024-04-21T10:30:03.303859+0000","last_peered":"2024-04-21T10:30:03.303859+0000","last_clean":"2024-04-21T10:30:03.303859+0000","last_became_active":"2024-04-21T10:29:01.942151+0000","last_became_peered":"2024-04-21T10:29:01.942151+0000","last_unstale":"2024-04-21T10:30:03.303859+0000","last_undegraded":"2024-04-21T10:30:03.303859+0000","last_fullsized":"2024-04-21T10:30:03.303859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:39.071158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305052+0000","last_change":"2024-04-21T10:29:00.928221+0000","last_active":"2024-04-21T10:30:03.305052+0000","last_peered":"2024-04-21T10:30:03.305052+0000","last_clean":"2024-04-21T10:30:03.305052+0000","last_became_active":"2024-04-21T10:29:00.927873+0000","last_became_peered":"2024-04-21T10:29:00.927873+0000","last_unstale":"2024-04-21T10:30:03.305052+0000","last_undegraded":"2024-04-21T10:30:03.305052+0000","last_fullsized":"2024-04-21T10:30:03.305052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:14.898456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301059+0000","last_change":"2024-04-21T10:29:01.930375+0000","last_active":"2024-04-21T10:30:03.301059+0000","last_peered":"2024-04-21T10:30:03.301059+0000","last_clean":"2024-04-21T10:30:03.301059+0000","last_became_active":"2024-04-21T10:29:01.930009+0000","last_became_peered":"2024-04-21T10:29:01.930009+0000","last_unstale":"2024-04-21T10:30:03.301059+0000","last_undegraded":"2024-04-21T10:30:03.301059+0000","last_fullsized":"2024-04-21T10:30:03.301059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:34.111797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:10.910233+0000","last_change":"2024-04-21T10:29:00.922981+0000","last_active":"2024-04-21T10:30:10.910233+0000","last_peered":"2024-04-21T10:30:10.910233+0000","last_clean":"2024-04-21T10:30:10.910233+0000","last_became_active":"2024-04-21T10:29:00.922668+0000","last_became_peered":"2024-04-21T10:29:00.922668+0000","last_unstale":"2024-04-21T10:30:10.910233+0000","last_undegraded":"2024-04-21T10:30:10.910233+0000","last_fullsized":"2024-04-21T10:30:10.910233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:47.109375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300763+0000","last_change":"2024-04-21T10:29:01.942530+0000","last_active":"2024-04-21T10:30:03.300763+0000","last_peered":"2024-04-21T10:30:03.300763+0000","last_clean":"2024-04-21T10:30:03.300763+0000","last_became_active":"2024-04-21T10:29:01.942162+0000","last_became_peered":"2024-04-21T10:29:01.942162+0000","last_unstale":"2024-04-21T10:30:03.300763+0000","last_undegraded":"2024-04-21T10:30:03.300763+0000","last_fullsized":"2024-04-21T10:30:03.300763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:46:19.138930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.544712+0000","last_change":"2024-04-21T10:30:03.544712+0000","last_active":"2024-04-21T10:30:03.544712+0000","last_peered":"2024-04-21T10:30:03.544712+0000","last_clean":"2024-04-21T10:30:03.544712+0000","last_became_active":"2024-04-21T10:30:03.519730+0000","last_became_peered":"2024-04-21T10:30:03.519730+0000","last_unstale":"2024-04-21T10:30:03.544712+0000","last_undegraded":"2024-04-21T10:30:03.544712+0000","last_fullsized":"2024-04-21T10:30:03.544712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:54:33.184653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302388+0000","last_change":"2024-04-21T10:29:01.929696+0000","last_active":"2024-04-21T10:30:03.302388+0000","last_peered":"2024-04-21T10:30:03.302388+0000","last_clean":"2024-04-21T10:30:03.302388+0000","last_became_active":"2024-04-21T10:29:01.929328+0000","last_became_peered":"2024-04-21T10:29:01.929328+0000","last_unstale":"2024-04-21T10:30:03.302388+0000","last_undegraded":"2024-04-21T10:30:03.302388+0000","last_fullsized":"2024-04-21T10:30:03.302388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:55:42.735595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302375+0000","last_change":"2024-04-21T10:29:00.925981+0000","last_active":"2024-04-21T10:30:03.302375+0000","last_peered":"2024-04-21T10:30:03.302375+0000","last_clean":"2024-04-21T10:30:03.302375+0000","last_became_active":"2024-04-21T10:29:00.925560+0000","last_became_peered":"2024-04-21T10:29:00.925560+0000","last_unstale":"2024-04-21T10:30:03.302375+0000","last_undegraded":"2024-04-21T10:30:03.302375+0000","last_fullsized":"2024-04-21T10:30:03.302375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:37.467522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303410+0000","last_change":"2024-04-21T10:29:01.941961+0000","last_active":"2024-04-21T10:30:03.303410+0000","last_peered":"2024-04-21T10:30:03.303410+0000","last_clean":"2024-04-21T10:30:03.303410+0000","last_became_active":"2024-04-21T10:29:01.941649+0000","last_became_peered":"2024-04-21T10:29:01.941649+0000","last_unstale":"2024-04-21T10:30:03.303410+0000","last_undegraded":"2024-04-21T10:30:03.303410+0000","last_fullsized":"2024-04-21T10:30:03.303410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:00:56.803856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.299991+0000","last_change":"2024-04-21T10:29:00.929504+0000","last_active":"2024-04-21T10:30:03.299991+0000","last_peered":"2024-04-21T10:30:03.299991+0000","last_clean":"2024-04-21T10:30:03.299991+0000","last_became_active":"2024-04-21T10:29:00.929172+0000","last_became_peered":"2024-04-21T10:29:00.929172+0000","last_unstale":"2024-04-21T10:30:03.299991+0000","last_undegraded":"2024-04-21T10:30:03.299991+0000","last_fullsized":"2024-04-21T10:30:03.299991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:40.512175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303818+0000","last_change":"2024-04-21T10:29:01.939333+0000","last_active":"2024-04-21T10:30:03.303818+0000","last_peered":"2024-04-21T10:30:03.303818+0000","last_clean":"2024-04-21T10:30:03.303818+0000","last_became_active":"2024-04-21T10:29:01.938924+0000","last_became_peered":"2024-04-21T10:29:01.938924+0000","last_unstale":"2024-04-21T10:30:03.303818+0000","last_undegraded":"2024-04-21T10:30:03.303818+0000","last_fullsized":"2024-04-21T10:30:03.303818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:20:01.287604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300883+0000","last_change":"2024-04-21T10:29:00.927455+0000","last_active":"2024-04-21T10:30:03.300883+0000","last_peered":"2024-04-21T10:30:03.300883+0000","last_clean":"2024-04-21T10:30:03.300883+0000","last_became_active":"2024-04-21T10:29:00.927180+0000","last_became_peered":"2024-04-21T10:29:00.927180+0000","last_unstale":"2024-04-21T10:30:03.300883+0000","last_undegraded":"2024-04-21T10:30:03.300883+0000","last_fullsized":"2024-04-21T10:30:03.300883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:11:57.166607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303587+0000","last_change":"2024-04-21T10:29:01.937646+0000","last_active":"2024-04-21T10:30:03.303587+0000","last_peered":"2024-04-21T10:30:03.303587+0000","last_clean":"2024-04-21T10:30:03.303587+0000","last_became_active":"2024-04-21T10:29:01.937216+0000","last_became_peered":"2024-04-21T10:29:01.937216+0000","last_unstale":"2024-04-21T10:30:03.303587+0000","last_undegraded":"2024-04-21T10:30:03.303587+0000","last_fullsized":"2024-04-21T10:30:03.303587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:38.265252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305317+0000","last_change":"2024-04-21T10:29:01.926860+0000","last_active":"2024-04-21T10:30:03.305317+0000","last_peered":"2024-04-21T10:30:03.305317+0000","last_clean":"2024-04-21T10:30:03.305317+0000","last_became_active":"2024-04-21T10:29:01.925951+0000","last_became_peered":"2024-04-21T10:29:01.925951+0000","last_unstale":"2024-04-21T10:30:03.305317+0000","last_undegraded":"2024-04-21T10:30:03.305317+0000","last_fullsized":"2024-04-21T10:30:03.305317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:01:55.127301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:09.908483+0000","last_change":"2024-04-21T10:29:00.926502+0000","last_active":"2024-04-21T10:30:09.908483+0000","last_peered":"2024-04-21T10:30:09.908483+0000","last_clean":"2024-04-21T10:30:09.908483+0000","last_became_active":"2024-04-21T10:29:00.926175+0000","last_became_peered":"2024-04-21T10:29:00.926175+0000","last_unstale":"2024-04-21T10:30:09.908483+0000","last_undegraded":"2024-04-21T10:30:09.908483+0000","last_fullsized":"2024-04-21T10:30:09.908483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:40:34.774085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.520292+0000","last_change":"2024-04-21T10:30:03.519801+0000","last_active":"2024-04-21T10:30:03.520292+0000","last_peered":"2024-04-21T10:30:03.520292+0000","last_clean":"2024-04-21T10:30:03.520292+0000","last_became_active":"2024-04-21T10:30:03.519351+0000","last_became_peered":"2024-04-21T10:30:03.519351+0000","last_unstale":"2024-04-21T10:30:03.520292+0000","last_undegraded":"2024-04-21T10:30:03.520292+0000","last_fullsized":"2024-04-21T10:30:03.520292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:13:36.675062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518520+0000","last_change":"2024-04-21T10:29:00.921748+0000","last_active":"2024-04-21T10:30:03.518520+0000","last_peered":"2024-04-21T10:30:03.518520+0000","last_clean":"2024-04-21T10:30:03.518520+0000","last_became_active":"2024-04-21T10:29:00.921408+0000","last_became_peered":"2024-04-21T10:29:00.921408+0000","last_unstale":"2024-04-21T10:30:03.518520+0000","last_undegraded":"2024-04-21T10:30:03.518520+0000","last_fullsized":"2024-04-21T10:30:03.518520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:52:27.606280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300849+0000","last_change":"2024-04-21T10:27:49.787009+0000","last_active":"2024-04-21T10:30:03.300849+0000","last_peered":"2024-04-21T10:30:03.300849+0000","last_clean":"2024-04-21T10:30:03.300849+0000","last_became_active":"2024-04-21T10:27:49.786530+0000","last_became_peered":"2024-04-21T10:27:49.786530+0000","last_unstale":"2024-04-21T10:30:03.300849+0000","last_undegraded":"2024-04-21T10:30:03.300849+0000","last_fullsized":"2024-04-21T10:30:03.300849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_clean_scrub_stamp":"2024-04-21T10:24:28.367927+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:20:12.106231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305720+0000","last_change":"2024-04-21T10:30:03.305720+0000","last_active":"2024-04-21T10:30:03.305720+0000","last_peered":"2024-04-21T10:30:03.305720+0000","last_clean":"2024-04-21T10:30:03.305720+0000","last_became_active":"2024-04-21T10:30:03.305368+0000","last_became_peered":"2024-04-21T10:30:03.305368+0000","last_unstale":"2024-04-21T10:30:03.305720+0000","last_undegraded":"2024-04-21T10:30:03.305720+0000","last_fullsized":"2024-04-21T10:30:03.305720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:16.668743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300951+0000","last_change":"2024-04-21T10:29:00.927521+0000","last_active":"2024-04-21T10:30:03.300951+0000","last_peered":"2024-04-21T10:30:03.300951+0000","last_clean":"2024-04-21T10:30:03.300951+0000","last_became_active":"2024-04-21T10:29:00.927202+0000","last_became_peered":"2024-04-21T10:29:00.927202+0000","last_unstale":"2024-04-21T10:30:03.300951+0000","last_undegraded":"2024-04-21T10:30:03.300951+0000","last_fullsized":"2024-04-21T10:30:03.300951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:11:32.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302357+0000","last_change":"2024-04-21T10:29:00.930728+0000","last_active":"2024-04-21T10:30:03.302357+0000","last_peered":"2024-04-21T10:30:03.302357+0000","last_clean":"2024-04-21T10:30:03.302357+0000","last_became_active":"2024-04-21T10:29:00.930457+0000","last_became_peered":"2024-04-21T10:29:00.930457+0000","last_unstale":"2024-04-21T10:30:03.302357+0000","last_undegraded":"2024-04-21T10:30:03.302357+0000","last_fullsized":"2024-04-21T10:30:03.302357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:28.032954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.304625+0000","last_change":"2024-04-21T10:29:01.932278+0000","last_active":"2024-04-21T10:30:03.304625+0000","last_peered":"2024-04-21T10:30:03.304625+0000","last_clean":"2024-04-21T10:30:03.304625+0000","last_became_active":"2024-04-21T10:29:01.931774+0000","last_became_peered":"2024-04-21T10:29:01.931774+0000","last_unstale":"2024-04-21T10:30:03.304625+0000","last_undegraded":"2024-04-21T10:30:03.304625+0000","last_fullsized":"2024-04-21T10:30:03.304625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:34:25.858475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300194+0000","last_change":"2024-04-21T10:29:00.926071+0000","last_active":"2024-04-21T10:30:03.300194+0000","last_peered":"2024-04-21T10:30:03.300194+0000","last_clean":"2024-04-21T10:30:03.300194+0000","last_became_active":"2024-04-21T10:29:00.925607+0000","last_became_peered":"2024-04-21T10:29:00.925607+0000","last_unstale":"2024-04-21T10:30:03.300194+0000","last_undegraded":"2024-04-21T10:30:03.300194+0000","last_fullsized":"2024-04-21T10:30:03.300194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:26.350652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308249+0000","last_change":"2024-04-21T10:29:01.941365+0000","last_active":"2024-04-21T10:30:03.308249+0000","last_peered":"2024-04-21T10:30:03.308249+0000","last_clean":"2024-04-21T10:30:03.308249+0000","last_became_active":"2024-04-21T10:29:01.940102+0000","last_became_peered":"2024-04-21T10:29:01.940102+0000","last_unstale":"2024-04-21T10:30:03.308249+0000","last_undegraded":"2024-04-21T10:30:03.308249+0000","last_fullsized":"2024-04-21T10:30:03.308249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:21.976855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.519407+0000","last_change":"2024-04-21T10:29:00.924322+0000","last_active":"2024-04-21T10:30:03.519407+0000","last_peered":"2024-04-21T10:30:03.519407+0000","last_clean":"2024-04-21T10:30:03.519407+0000","last_became_active":"2024-04-21T10:29:00.924016+0000","last_became_peered":"2024-04-21T10:29:00.924016+0000","last_unstale":"2024-04-21T10:30:03.519407+0000","last_undegraded":"2024-04-21T10:30:03.519407+0000","last_fullsized":"2024-04-21T10:30:03.519407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:49:24.004951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308135+0000","last_change":"2024-04-21T10:29:01.946315+0000","last_active":"2024-04-21T10:30:03.308135+0000","last_peered":"2024-04-21T10:30:03.308135+0000","last_clean":"2024-04-21T10:30:03.308135+0000","last_became_active":"2024-04-21T10:29:01.945920+0000","last_became_peered":"2024-04-21T10:29:01.945920+0000","last_unstale":"2024-04-21T10:30:03.308135+0000","last_undegraded":"2024-04-21T10:30:03.308135+0000","last_fullsized":"2024-04-21T10:30:03.308135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:10:52.361615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518803+0000","last_change":"2024-04-21T10:29:00.919086+0000","last_active":"2024-04-21T10:30:03.518803+0000","last_peered":"2024-04-21T10:30:03.518803+0000","last_clean":"2024-04-21T10:30:03.518803+0000","last_became_active":"2024-04-21T10:29:00.918727+0000","last_became_peered":"2024-04-21T10:29:00.918727+0000","last_unstale":"2024-04-21T10:30:03.518803+0000","last_undegraded":"2024-04-21T10:30:03.518803+0000","last_fullsized":"2024-04-21T10:30:03.518803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:53:27.209135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301802+0000","last_change":"2024-04-21T10:29:01.944037+0000","last_active":"2024-04-21T10:30:03.301802+0000","last_peered":"2024-04-21T10:30:03.301802+0000","last_clean":"2024-04-21T10:30:03.301802+0000","last_became_active":"2024-04-21T10:29:01.943692+0000","last_became_peered":"2024-04-21T10:29:01.943692+0000","last_unstale":"2024-04-21T10:30:03.301802+0000","last_undegraded":"2024-04-21T10:30:03.301802+0000","last_fullsized":"2024-04-21T10:30:03.301802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:51.285535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307485+0000","last_change":"2024-04-21T10:29:00.930937+0000","last_active":"2024-04-21T10:30:03.307485+0000","last_peered":"2024-04-21T10:30:03.307485+0000","last_clean":"2024-04-21T10:30:03.307485+0000","last_became_active":"2024-04-21T10:29:00.930348+0000","last_became_peered":"2024-04-21T10:29:00.930348+0000","last_unstale":"2024-04-21T10:30:03.307485+0000","last_undegraded":"2024-04-21T10:30:03.307485+0000","last_fullsized":"2024-04-21T10:30:03.307485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:43:33.801432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.304513+0000","last_change":"2024-04-21T10:29:01.937116+0000","last_active":"2024-04-21T10:30:03.304513+0000","last_peered":"2024-04-21T10:30:03.304513+0000","last_clean":"2024-04-21T10:30:03.304513+0000","last_became_active":"2024-04-21T10:29:01.936756+0000","last_became_peered":"2024-04-21T10:29:01.936756+0000","last_unstale":"2024-04-21T10:30:03.304513+0000","last_undegraded":"2024-04-21T10:30:03.304513+0000","last_fullsized":"2024-04-21T10:30:03.304513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:02.732399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.312469+0000","last_change":"2024-04-21T10:30:03.312469+0000","last_active":"2024-04-21T10:30:03.312469+0000","last_peered":"2024-04-21T10:30:03.312469+0000","last_clean":"2024-04-21T10:30:03.312469+0000","last_became_active":"2024-04-21T10:30:03.310027+0000","last_became_peered":"2024-04-21T10:30:03.310027+0000","last_unstale":"2024-04-21T10:30:03.312469+0000","last_undegraded":"2024-04-21T10:30:03.312469+0000","last_fullsized":"2024-04-21T10:30:03.312469+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:12:18.582539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308475+0000","last_change":"2024-04-21T10:29:01.940677+0000","last_active":"2024-04-21T10:30:03.308475+0000","last_peered":"2024-04-21T10:30:03.308475+0000","last_clean":"2024-04-21T10:30:03.308475+0000","last_became_active":"2024-04-21T10:29:01.940296+0000","last_became_peered":"2024-04-21T10:29:01.940296+0000","last_unstale":"2024-04-21T10:30:03.308475+0000","last_undegraded":"2024-04-21T10:30:03.308475+0000","last_fullsized":"2024-04-21T10:30:03.308475+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:24:52.662041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302074+0000","last_change":"2024-04-21T10:29:00.924392+0000","last_active":"2024-04-21T10:30:03.302074+0000","last_peered":"2024-04-21T10:30:03.302074+0000","last_clean":"2024-04-21T10:30:03.302074+0000","last_became_active":"2024-04-21T10:29:00.924029+0000","last_became_peered":"2024-04-21T10:29:00.924029+0000","last_unstale":"2024-04-21T10:30:03.302074+0000","last_undegraded":"2024-04-21T10:30:03.302074+0000","last_fullsized":"2024-04-21T10:30:03.302074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:23:50.726978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301081+0000","last_change":"2024-04-21T10:29:01.939687+0000","last_active":"2024-04-21T10:30:03.301081+0000","last_peered":"2024-04-21T10:30:03.301081+0000","last_clean":"2024-04-21T10:30:03.301081+0000","last_became_active":"2024-04-21T10:29:01.939343+0000","last_became_peered":"2024-04-21T10:29:01.939343+0000","last_unstale":"2024-04-21T10:30:03.301081+0000","last_undegraded":"2024-04-21T10:30:03.301081+0000","last_fullsized":"2024-04-21T10:30:03.301081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:50.481998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301786+0000","last_change":"2024-04-21T10:29:00.923115+0000","last_active":"2024-04-21T10:30:03.301786+0000","last_peered":"2024-04-21T10:30:03.301786+0000","last_clean":"2024-04-21T10:30:03.301786+0000","last_became_active":"2024-04-21T10:29:00.922707+0000","last_became_peered":"2024-04-21T10:29:00.922707+0000","last_unstale":"2024-04-21T10:30:03.301786+0000","last_undegraded":"2024-04-21T10:30:03.301786+0000","last_fullsized":"2024-04-21T10:30:03.301786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:55:53.419904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301150+0000","last_change":"2024-04-21T10:29:01.940666+0000","last_active":"2024-04-21T10:30:03.301150+0000","last_peered":"2024-04-21T10:30:03.301150+0000","last_clean":"2024-04-21T10:30:03.301150+0000","last_became_active":"2024-04-21T10:29:01.940331+0000","last_became_peered":"2024-04-21T10:29:01.940331+0000","last_unstale":"2024-04-21T10:30:03.301150+0000","last_undegraded":"2024-04-21T10:30:03.301150+0000","last_fullsized":"2024-04-21T10:30:03.301150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:52:41.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305749+0000","last_change":"2024-04-21T10:29:00.927873+0000","last_active":"2024-04-21T10:30:03.305749+0000","last_peered":"2024-04-21T10:30:03.305749+0000","last_clean":"2024-04-21T10:30:03.305749+0000","last_became_active":"2024-04-21T10:29:00.927403+0000","last_became_peered":"2024-04-21T10:29:00.927403+0000","last_unstale":"2024-04-21T10:30:03.305749+0000","last_undegraded":"2024-04-21T10:30:03.305749+0000","last_fullsized":"2024-04-21T10:30:03.305749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:05:05.276978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301019+0000","last_change":"2024-04-21T10:29:01.944725+0000","last_active":"2024-04-21T10:30:03.301019+0000","last_peered":"2024-04-21T10:30:03.301019+0000","last_clean":"2024-04-21T10:30:03.301019+0000","last_became_active":"2024-04-21T10:29:01.944326+0000","last_became_peered":"2024-04-21T10:29:01.944326+0000","last_unstale":"2024-04-21T10:30:03.301019+0000","last_undegraded":"2024-04-21T10:30:03.301019+0000","last_fullsized":"2024-04-21T10:30:03.301019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:50.209064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303976+0000","last_change":"2024-04-21T10:29:00.924694+0000","last_active":"2024-04-21T10:30:03.303976+0000","last_peered":"2024-04-21T10:30:03.303976+0000","last_clean":"2024-04-21T10:30:03.303976+0000","last_became_active":"2024-04-21T10:29:00.924373+0000","last_became_peered":"2024-04-21T10:29:00.924373+0000","last_unstale":"2024-04-21T10:30:03.303976+0000","last_undegraded":"2024-04-21T10:30:03.303976+0000","last_fullsized":"2024-04-21T10:30:03.303976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:10:14.681630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301133+0000","last_change":"2024-04-21T10:29:01.939277+0000","last_active":"2024-04-21T10:30:03.301133+0000","last_peered":"2024-04-21T10:30:03.301133+0000","last_clean":"2024-04-21T10:30:03.301133+0000","last_became_active":"2024-04-21T10:29:01.938844+0000","last_became_peered":"2024-04-21T10:29:01.938844+0000","last_unstale":"2024-04-21T10:30:03.301133+0000","last_undegraded":"2024-04-21T10:30:03.301133+0000","last_fullsized":"2024-04-21T10:30:03.301133+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:38:14.553868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.304616+0000","last_change":"2024-04-21T10:29:01.942654+0000","last_active":"2024-04-21T10:30:03.304616+0000","last_peered":"2024-04-21T10:30:03.304616+0000","last_clean":"2024-04-21T10:30:03.304616+0000","last_became_active":"2024-04-21T10:29:01.942324+0000","last_became_peered":"2024-04-21T10:29:01.942324+0000","last_unstale":"2024-04-21T10:30:03.304616+0000","last_undegraded":"2024-04-21T10:30:03.304616+0000","last_fullsized":"2024-04-21T10:30:03.304616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:19.027138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307328+0000","last_change":"2024-04-21T10:30:03.307328+0000","last_active":"2024-04-21T10:30:03.307328+0000","last_peered":"2024-04-21T10:30:03.307328+0000","last_clean":"2024-04-21T10:30:03.307328+0000","last_became_active":"2024-04-21T10:30:03.306976+0000","last_became_peered":"2024-04-21T10:30:03.306976+0000","last_unstale":"2024-04-21T10:30:03.307328+0000","last_undegraded":"2024-04-21T10:30:03.307328+0000","last_fullsized":"2024-04-21T10:30:03.307328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:19:31.739372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301467+0000","last_change":"2024-04-21T10:29:01.944405+0000","last_active":"2024-04-21T10:30:03.301467+0000","last_peered":"2024-04-21T10:30:03.301467+0000","last_clean":"2024-04-21T10:30:03.301467+0000","last_became_active":"2024-04-21T10:29:01.944085+0000","last_became_peered":"2024-04-21T10:29:01.944085+0000","last_unstale":"2024-04-21T10:30:03.301467+0000","last_undegraded":"2024-04-21T10:30:03.301467+0000","last_fullsized":"2024-04-21T10:30:03.301467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:13.299859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307414+0000","last_change":"2024-04-21T10:29:01.941054+0000","last_active":"2024-04-21T10:30:03.307414+0000","last_peered":"2024-04-21T10:30:03.307414+0000","last_clean":"2024-04-21T10:30:03.307414+0000","last_became_active":"2024-04-21T10:29:01.940243+0000","last_became_peered":"2024-04-21T10:29:01.940243+0000","last_unstale":"2024-04-21T10:30:03.307414+0000","last_undegraded":"2024-04-21T10:30:03.307414+0000","last_fullsized":"2024-04-21T10:30:03.307414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:31:04.222026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307984+0000","last_change":"2024-04-21T10:30:03.307984+0000","last_active":"2024-04-21T10:30:03.307984+0000","last_peered":"2024-04-21T10:30:03.307984+0000","last_clean":"2024-04-21T10:30:03.307984+0000","last_became_active":"2024-04-21T10:30:03.307584+0000","last_became_peered":"2024-04-21T10:30:03.307584+0000","last_unstale":"2024-04-21T10:30:03.307984+0000","last_undegraded":"2024-04-21T10:30:03.307984+0000","last_fullsized":"2024-04-21T10:30:03.307984+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:12.699665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303423+0000","last_change":"2024-04-21T10:29:01.947268+0000","last_active":"2024-04-21T10:30:03.303423+0000","last_peered":"2024-04-21T10:30:03.303423+0000","last_clean":"2024-04-21T10:30:03.303423+0000","last_became_active":"2024-04-21T10:29:01.946921+0000","last_became_peered":"2024-04-21T10:29:01.946921+0000","last_unstale":"2024-04-21T10:30:03.303423+0000","last_undegraded":"2024-04-21T10:30:03.303423+0000","last_fullsized":"2024-04-21T10:30:03.303423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:50:43.784920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302177+0000","last_change":"2024-04-21T10:29:01.949969+0000","last_active":"2024-04-21T10:30:03.302177+0000","last_peered":"2024-04-21T10:30:03.302177+0000","last_clean":"2024-04-21T10:30:03.302177+0000","last_became_active":"2024-04-21T10:29:01.949487+0000","last_became_peered":"2024-04-21T10:29:01.949487+0000","last_unstale":"2024-04-21T10:30:03.302177+0000","last_undegraded":"2024-04-21T10:30:03.302177+0000","last_fullsized":"2024-04-21T10:30:03.302177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:26:59.620209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308151+0000","last_change":"2024-04-21T10:29:01.937904+0000","last_active":"2024-04-21T10:30:03.308151+0000","last_peered":"2024-04-21T10:30:03.308151+0000","last_clean":"2024-04-21T10:30:03.308151+0000","last_became_active":"2024-04-21T10:29:01.937504+0000","last_became_peered":"2024-04-21T10:29:01.937504+0000","last_unstale":"2024-04-21T10:30:03.308151+0000","last_undegraded":"2024-04-21T10:30:03.308151+0000","last_fullsized":"2024-04-21T10:30:03.308151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:58:24.245118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.310042+0000","last_change":"2024-04-21T10:30:03.310042+0000","last_active":"2024-04-21T10:30:03.310042+0000","last_peered":"2024-04-21T10:30:03.310042+0000","last_clean":"2024-04-21T10:30:03.310042+0000","last_became_active":"2024-04-21T10:30:03.309663+0000","last_became_peered":"2024-04-21T10:30:03.309663+0000","last_unstale":"2024-04-21T10:30:03.310042+0000","last_undegraded":"2024-04-21T10:30:03.310042+0000","last_fullsized":"2024-04-21T10:30:03.310042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:19.906949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307387+0000","last_change":"2024-04-21T10:29:01.936880+0000","last_active":"2024-04-21T10:30:03.307387+0000","last_peered":"2024-04-21T10:30:03.307387+0000","last_clean":"2024-04-21T10:30:03.307387+0000","last_became_active":"2024-04-21T10:29:01.936555+0000","last_became_peered":"2024-04-21T10:29:01.936555+0000","last_unstale":"2024-04-21T10:30:03.307387+0000","last_undegraded":"2024-04-21T10:30:03.307387+0000","last_fullsized":"2024-04-21T10:30:03.307387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:32.831606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518462+0000","last_change":"2024-04-21T10:29:01.943238+0000","last_active":"2024-04-21T10:30:03.518462+0000","last_peered":"2024-04-21T10:30:03.518462+0000","last_clean":"2024-04-21T10:30:03.518462+0000","last_became_active":"2024-04-21T10:29:01.942841+0000","last_became_peered":"2024-04-21T10:29:01.942841+0000","last_unstale":"2024-04-21T10:30:03.518462+0000","last_undegraded":"2024-04-21T10:30:03.518462+0000","last_fullsized":"2024-04-21T10:30:03.518462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:59:48.551524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303779+0000","last_change":"2024-04-21T10:29:01.933712+0000","last_active":"2024-04-21T10:30:03.303779+0000","last_peered":"2024-04-21T10:30:03.303779+0000","last_clean":"2024-04-21T10:30:03.303779+0000","last_became_active":"2024-04-21T10:29:01.933399+0000","last_became_peered":"2024-04-21T10:29:01.933399+0000","last_unstale":"2024-04-21T10:30:03.303779+0000","last_undegraded":"2024-04-21T10:30:03.303779+0000","last_fullsized":"2024-04-21T10:30:03.303779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:34:16.346403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303235+0000","last_change":"2024-04-21T10:29:01.947225+0000","last_active":"2024-04-21T10:30:03.303235+0000","last_peered":"2024-04-21T10:30:03.303235+0000","last_clean":"2024-04-21T10:30:03.303235+0000","last_became_active":"2024-04-21T10:29:01.946890+0000","last_became_peered":"2024-04-21T10:29:01.946890+0000","last_unstale":"2024-04-21T10:30:03.303235+0000","last_undegraded":"2024-04-21T10:30:03.303235+0000","last_fullsized":"2024-04-21T10:30:03.303235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:19:00.960767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64024,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":428,"num_read_kb":281,"num_write":158,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":192072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318301,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":463981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579937,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1115764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":464105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005099,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":463913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266735,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1055955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430261,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":463494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":523422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":464765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":524548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":463805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888128,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":978944,"data_stored":464322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691899,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1593344,"data_stored":1055248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T10:30:15.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-21T10:30:16.214 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:16.557 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:16 smithi190 ceph-mon[31423]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:30:16.557 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:16 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:30:16.557 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:16 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:30:16.557 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:16 smithi190 ceph-mon[31423]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:16.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:16 smithi026 ceph-mon[27965]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:30:16.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:16 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:30:16.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:16 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:30:16.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:16 smithi026 ceph-mon[27965]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:16.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:16 smithi169 ceph-mon[31469]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:30:16.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:16 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:30:16.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:16 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:30:16.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:16 smithi169 ceph-mon[31469]: from='mgr.14152 172.21.15.26:0/3771349084' entity='mgr.x' 2024-04-21T10:30:17.606 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:17 smithi026 ceph-mon[27965]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-04-21T10:30:17.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:17 smithi169 ceph-mon[31469]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-04-21T10:30:17.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:17 smithi190 ceph-mon[31423]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-04-21T10:30:17.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.816+0000 7f51ab4e2700 1 -- 172.21.15.26:0/1529458339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a4102320 msgr2=0x7f51a410e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:17.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.816+0000 7f51ab4e2700 1 --2- 172.21.15.26:0/1529458339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a4102320 0x7f51a410e830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f519c00b0a0 tx=0x7f519c00b3b0 comp rx=0 tx=0).stop 2024-04-21T10:30:17.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.816+0000 7f51ab4e2700 1 -- 172.21.15.26:0/1529458339 shutdown_connections 2024-04-21T10:30:17.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.816+0000 7f51ab4e2700 1 --2- 172.21.15.26:0/1529458339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a4107960 0x7f51a4107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:17.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.816+0000 7f51ab4e2700 1 --2- 172.21.15.26:0/1529458339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a4102320 0x7f51a410e830 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:17.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.816+0000 7f51ab4e2700 1 --2- 172.21.15.26:0/1529458339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4101960 0x7f51a4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:17.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.816+0000 7f51ab4e2700 1 -- 172.21.15.26:0/1529458339 >> 172.21.15.26:0/1529458339 conn(0x7f51a40789e0 msgr2=0x7f51a4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:17.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.816+0000 7f51ab4e2700 1 -- 172.21.15.26:0/1529458339 shutdown_connections 2024-04-21T10:30:17.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.816+0000 7f51ab4e2700 1 -- 172.21.15.26:0/1529458339 wait complete. 2024-04-21T10:30:17.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.816+0000 7f51ab4e2700 1 Processor -- start 2024-04-21T10:30:17.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.816+0000 7f51ab4e2700 1 -- start start 2024-04-21T10:30:17.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51ab4e2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4101960 0x7f51a41991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:17.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51ab4e2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a4102320 0x7f51a4199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:17.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51ab4e2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a4107960 0x7f51a419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51ab4e2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4111ba0 con 0x7f51a4102320 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51ab4e2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4111a20 con 0x7f51a4107960 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51ab4e2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4111d20 con 0x7f51a4101960 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51a8a7d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a4102320 0x7f51a4199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51a8a7d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a4102320 0x7f51a4199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47782/0 (socket says 172.21.15.26:47782) 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51a8a7d700 1 -- 172.21.15.26:0/3677172353 learned_addr learned my addr 172.21.15.26:0/3677172353 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51a927e700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4101960 0x7f51a41991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51a9a7f700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a4107960 0x7f51a419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51a8a7d700 1 -- 172.21.15.26:0/3677172353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4101960 msgr2=0x7f51a41991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51a8a7d700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4101960 0x7f51a41991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51a8a7d700 1 -- 172.21.15.26:0/3677172353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a4107960 msgr2=0x7f51a419da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51a8a7d700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a4107960 0x7f51a419da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51a8a7d700 1 -- 172.21.15.26:0/3677172353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a419e1d0 con 0x7f51a4102320 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51a8a7d700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a4102320 0x7f51a4199700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f519400d980 tx=0x7f519400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:17.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f519a7fc700 1 -- 172.21.15.26:0/3677172353 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194017070 con 0x7f51a4102320 2024-04-21T10:30:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f519a7fc700 1 -- 172.21.15.26:0/3677172353 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5194005020 con 0x7f51a4102320 2024-04-21T10:30:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f519a7fc700 1 -- 172.21.15.26:0/3677172353 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194009b50 con 0x7f51a4102320 2024-04-21T10:30:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.817+0000 7f51ab4e2700 1 -- 172.21.15.26:0/3677172353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a419e4c0 con 0x7f51a4102320 2024-04-21T10:30:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.818+0000 7f51ab4e2700 1 -- 172.21.15.26:0/3677172353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41a5d40 con 0x7f51a4102320 2024-04-21T10:30:17.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.819+0000 7f51ab4e2700 1 -- 172.21.15.26:0/3677172353 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a4069730 con 0x7f51a4102320 2024-04-21T10:30:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.820+0000 7f519a7fc700 1 -- 172.21.15.26:0/3677172353 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f51940051d0 con 0x7f51a4102320 2024-04-21T10:30:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.820+0000 7f519a7fc700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f51900a3d50 0x7f51900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.820+0000 7f51a927e700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f51900a3d50 0x7f51900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.821+0000 7f519a7fc700 1 -- 172.21.15.26:0/3677172353 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f51940d0820 con 0x7f51a4102320 2024-04-21T10:30:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.821+0000 7f51a927e700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f51900a3d50 0x7f51900a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009a10 tx=0x7f51a0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:17.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:17.824+0000 7f519a7fc700 1 -- 172.21.15.26:0/3677172353 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5194020020 con 0x7f51a4102320 2024-04-21T10:30:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.028+0000 7f51ab4e2700 1 -- 172.21.15.26:0/3677172353 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51a4062e80 con 0x7f51900a3d50 2024-04-21T10:30:18.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.035+0000 7f519a7fc700 1 -- 172.21.15.26:0/3677172353 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258722 (secure 0 0 0) 0x7f51a4062e80 con 0x7f51900a3d50 2024-04-21T10:30:18.038 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.038+0000 7f51ab4e2700 1 -- 172.21.15.26:0/3677172353 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f51900a3d50 msgr2=0x7f51900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.038+0000 7f51ab4e2700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f51900a3d50 0x7f51900a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009a10 tx=0x7f51a0009190 comp rx=0 tx=0).stop 2024-04-21T10:30:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.038+0000 7f51ab4e2700 1 -- 172.21.15.26:0/3677172353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a4102320 msgr2=0x7f51a4199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.038+0000 7f51ab4e2700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a4102320 0x7f51a4199700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f519400d980 tx=0x7f519400dc90 comp rx=0 tx=0).stop 2024-04-21T10:30:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.038+0000 7f51ab4e2700 1 -- 172.21.15.26:0/3677172353 shutdown_connections 2024-04-21T10:30:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.038+0000 7f51ab4e2700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f51900a3d50 0x7f51900a6210 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.038+0000 7f51ab4e2700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51a4107960 0x7f51a419da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.038+0000 7f51ab4e2700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a4101960 0x7f51a41991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.038+0000 7f51ab4e2700 1 --2- 172.21.15.26:0/3677172353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a4102320 0x7f51a4199700 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.038+0000 7f51ab4e2700 1 -- 172.21.15.26:0/3677172353 >> 172.21.15.26:0/3677172353 conn(0x7f51a40789e0 msgr2=0x7f51a406b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.038+0000 7f51ab4e2700 1 -- 172.21.15.26:0/3677172353 shutdown_connections 2024-04-21T10:30:18.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:18.038+0000 7f51ab4e2700 1 -- 172.21.15.26:0/3677172353 wait complete. 2024-04-21T10:30:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-21T10:30:18.687 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-04-21T10:30:16.196383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654626,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":564,"num_read_kb":400,"num_write":341,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362716,"kb_used_data":13500,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579108,"statfs":{"total":1151940427776,"available":1151569006592,"internally_reserved":0,"allocated":13824000,"data_stored":7523322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-9,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003139"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303593+0000","last_change":"2024-04-21T10:29:01.940996+0000","last_active":"2024-04-21T10:30:03.303593+0000","last_peered":"2024-04-21T10:30:03.303593+0000","last_clean":"2024-04-21T10:30:03.303593+0000","last_became_active":"2024-04-21T10:29:01.940662+0000","last_became_peered":"2024-04-21T10:29:01.940662+0000","last_unstale":"2024-04-21T10:30:03.303593+0000","last_undegraded":"2024-04-21T10:30:03.303593+0000","last_fullsized":"2024-04-21T10:30:03.303593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:16:01.434895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308135+0000","last_change":"2024-04-21T10:29:01.939238+0000","last_active":"2024-04-21T10:30:03.308135+0000","last_peered":"2024-04-21T10:30:03.308135+0000","last_clean":"2024-04-21T10:30:03.308135+0000","last_became_active":"2024-04-21T10:29:01.938907+0000","last_became_peered":"2024-04-21T10:29:01.938907+0000","last_unstale":"2024-04-21T10:30:03.308135+0000","last_undegraded":"2024-04-21T10:30:03.308135+0000","last_fullsized":"2024-04-21T10:30:03.308135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:09.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307936+0000","last_change":"2024-04-21T10:29:01.932338+0000","last_active":"2024-04-21T10:30:03.307936+0000","last_peered":"2024-04-21T10:30:03.307936+0000","last_clean":"2024-04-21T10:30:03.307936+0000","last_became_active":"2024-04-21T10:29:01.931916+0000","last_became_peered":"2024-04-21T10:29:01.931916+0000","last_unstale":"2024-04-21T10:30:03.307936+0000","last_undegraded":"2024-04-21T10:30:03.307936+0000","last_fullsized":"2024-04-21T10:30:03.307936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:52.859695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.519199+0000","last_change":"2024-04-21T10:29:01.945572+0000","last_active":"2024-04-21T10:30:03.519199+0000","last_peered":"2024-04-21T10:30:03.519199+0000","last_clean":"2024-04-21T10:30:03.519199+0000","last_became_active":"2024-04-21T10:29:01.940114+0000","last_became_peered":"2024-04-21T10:29:01.940114+0000","last_unstale":"2024-04-21T10:30:03.519199+0000","last_undegraded":"2024-04-21T10:30:03.519199+0000","last_fullsized":"2024-04-21T10:30:03.519199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:21.935573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301983+0000","last_change":"2024-04-21T10:29:01.946021+0000","last_active":"2024-04-21T10:30:03.301983+0000","last_peered":"2024-04-21T10:30:03.301983+0000","last_clean":"2024-04-21T10:30:03.301983+0000","last_became_active":"2024-04-21T10:29:01.945637+0000","last_became_peered":"2024-04-21T10:29:01.945637+0000","last_unstale":"2024-04-21T10:30:03.301983+0000","last_undegraded":"2024-04-21T10:30:03.301983+0000","last_fullsized":"2024-04-21T10:30:03.301983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:20.840494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.304092+0000","last_change":"2024-04-21T10:29:01.935402+0000","last_active":"2024-04-21T10:30:03.304092+0000","last_peered":"2024-04-21T10:30:03.304092+0000","last_clean":"2024-04-21T10:30:03.304092+0000","last_became_active":"2024-04-21T10:29:01.934994+0000","last_became_peered":"2024-04-21T10:29:01.934994+0000","last_unstale":"2024-04-21T10:30:03.304092+0000","last_undegraded":"2024-04-21T10:30:03.304092+0000","last_fullsized":"2024-04-21T10:30:03.304092+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:18.320530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300878+0000","last_change":"2024-04-21T10:29:01.943185+0000","last_active":"2024-04-21T10:30:03.300878+0000","last_peered":"2024-04-21T10:30:03.300878+0000","last_clean":"2024-04-21T10:30:03.300878+0000","last_became_active":"2024-04-21T10:29:01.942805+0000","last_became_peered":"2024-04-21T10:29:01.942805+0000","last_unstale":"2024-04-21T10:30:03.300878+0000","last_undegraded":"2024-04-21T10:30:03.300878+0000","last_fullsized":"2024-04-21T10:30:03.300878+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:06.776472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307431+0000","last_change":"2024-04-21T10:29:01.940995+0000","last_active":"2024-04-21T10:30:03.307431+0000","last_peered":"2024-04-21T10:30:03.307431+0000","last_clean":"2024-04-21T10:30:03.307431+0000","last_became_active":"2024-04-21T10:29:01.939173+0000","last_became_peered":"2024-04-21T10:29:01.939173+0000","last_unstale":"2024-04-21T10:30:03.307431+0000","last_undegraded":"2024-04-21T10:30:03.307431+0000","last_fullsized":"2024-04-21T10:30:03.307431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.740147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.519792+0000","last_change":"2024-04-21T10:29:01.944284+0000","last_active":"2024-04-21T10:30:03.519792+0000","last_peered":"2024-04-21T10:30:03.519792+0000","last_clean":"2024-04-21T10:30:03.519792+0000","last_became_active":"2024-04-21T10:29:01.943965+0000","last_became_peered":"2024-04-21T10:29:01.943965+0000","last_unstale":"2024-04-21T10:30:03.519792+0000","last_undegraded":"2024-04-21T10:30:03.519792+0000","last_fullsized":"2024-04-21T10:30:03.519792+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:08:54.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303863+0000","last_change":"2024-04-21T10:29:01.936250+0000","last_active":"2024-04-21T10:30:03.303863+0000","last_peered":"2024-04-21T10:30:03.303863+0000","last_clean":"2024-04-21T10:30:03.303863+0000","last_became_active":"2024-04-21T10:29:01.935904+0000","last_became_peered":"2024-04-21T10:29:01.935904+0000","last_unstale":"2024-04-21T10:30:03.303863+0000","last_undegraded":"2024-04-21T10:30:03.303863+0000","last_fullsized":"2024-04-21T10:30:03.303863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:35.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301084+0000","last_change":"2024-04-21T10:29:01.946550+0000","last_active":"2024-04-21T10:30:03.301084+0000","last_peered":"2024-04-21T10:30:03.301084+0000","last_clean":"2024-04-21T10:30:03.301084+0000","last_became_active":"2024-04-21T10:29:01.946170+0000","last_became_peered":"2024-04-21T10:29:01.946170+0000","last_unstale":"2024-04-21T10:30:03.301084+0000","last_undegraded":"2024-04-21T10:30:03.301084+0000","last_fullsized":"2024-04-21T10:30:03.301084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:25:47.671821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.519100+0000","last_change":"2024-04-21T10:29:01.946036+0000","last_active":"2024-04-21T10:30:03.519100+0000","last_peered":"2024-04-21T10:30:03.519100+0000","last_clean":"2024-04-21T10:30:03.519100+0000","last_became_active":"2024-04-21T10:29:01.940705+0000","last_became_peered":"2024-04-21T10:29:01.940705+0000","last_unstale":"2024-04-21T10:30:03.519100+0000","last_undegraded":"2024-04-21T10:30:03.519100+0000","last_fullsized":"2024-04-21T10:30:03.519100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:11:33.196059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303710+0000","last_change":"2024-04-21T10:29:01.940438+0000","last_active":"2024-04-21T10:30:03.303710+0000","last_peered":"2024-04-21T10:30:03.303710+0000","last_clean":"2024-04-21T10:30:03.303710+0000","last_became_active":"2024-04-21T10:29:01.940085+0000","last_became_peered":"2024-04-21T10:29:01.940085+0000","last_unstale":"2024-04-21T10:30:03.303710+0000","last_undegraded":"2024-04-21T10:30:03.303710+0000","last_fullsized":"2024-04-21T10:30:03.303710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:22:47.828858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307189+0000","last_change":"2024-04-21T10:29:01.941667+0000","last_active":"2024-04-21T10:30:03.307189+0000","last_peered":"2024-04-21T10:30:03.307189+0000","last_clean":"2024-04-21T10:30:03.307189+0000","last_became_active":"2024-04-21T10:29:01.940745+0000","last_became_peered":"2024-04-21T10:29:01.940745+0000","last_unstale":"2024-04-21T10:30:03.307189+0000","last_undegraded":"2024-04-21T10:30:03.307189+0000","last_fullsized":"2024-04-21T10:30:03.307189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:45:51.825780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.306896+0000","last_change":"2024-04-21T10:29:01.947771+0000","last_active":"2024-04-21T10:30:03.306896+0000","last_peered":"2024-04-21T10:30:03.306896+0000","last_clean":"2024-04-21T10:30:03.306896+0000","last_became_active":"2024-04-21T10:29:01.946818+0000","last_became_peered":"2024-04-21T10:29:01.946818+0000","last_unstale":"2024-04-21T10:30:03.306896+0000","last_undegraded":"2024-04-21T10:30:03.306896+0000","last_fullsized":"2024-04-21T10:30:03.306896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:18:42.159152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518778+0000","last_change":"2024-04-21T10:29:01.942499+0000","last_active":"2024-04-21T10:30:03.518778+0000","last_peered":"2024-04-21T10:30:03.518778+0000","last_clean":"2024-04-21T10:30:03.518778+0000","last_became_active":"2024-04-21T10:29:01.942136+0000","last_became_peered":"2024-04-21T10:29:01.942136+0000","last_unstale":"2024-04-21T10:30:03.518778+0000","last_undegraded":"2024-04-21T10:30:03.518778+0000","last_fullsized":"2024-04-21T10:30:03.518778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:34:11.822230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303603+0000","last_change":"2024-04-21T10:29:01.948822+0000","last_active":"2024-04-21T10:30:03.303603+0000","last_peered":"2024-04-21T10:30:03.303603+0000","last_clean":"2024-04-21T10:30:03.303603+0000","last_became_active":"2024-04-21T10:29:01.948492+0000","last_became_peered":"2024-04-21T10:29:01.948492+0000","last_unstale":"2024-04-21T10:30:03.303603+0000","last_undegraded":"2024-04-21T10:30:03.303603+0000","last_fullsized":"2024-04-21T10:30:03.303603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:06.992585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300138+0000","last_change":"2024-04-21T10:29:01.931265+0000","last_active":"2024-04-21T10:30:03.300138+0000","last_peered":"2024-04-21T10:30:03.300138+0000","last_clean":"2024-04-21T10:30:03.300138+0000","last_became_active":"2024-04-21T10:29:01.930877+0000","last_became_peered":"2024-04-21T10:29:01.930877+0000","last_unstale":"2024-04-21T10:30:03.300138+0000","last_undegraded":"2024-04-21T10:30:03.300138+0000","last_fullsized":"2024-04-21T10:30:03.300138+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:49:00.951226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301987+0000","last_change":"2024-04-21T10:29:01.941718+0000","last_active":"2024-04-21T10:30:03.301987+0000","last_peered":"2024-04-21T10:30:03.301987+0000","last_clean":"2024-04-21T10:30:03.301987+0000","last_became_active":"2024-04-21T10:29:01.941386+0000","last_became_peered":"2024-04-21T10:29:01.941386+0000","last_unstale":"2024-04-21T10:30:03.301987+0000","last_undegraded":"2024-04-21T10:30:03.301987+0000","last_fullsized":"2024-04-21T10:30:03.301987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:51:29.847961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300070+0000","last_change":"2024-04-21T10:29:00.923670+0000","last_active":"2024-04-21T10:30:03.300070+0000","last_peered":"2024-04-21T10:30:03.300070+0000","last_clean":"2024-04-21T10:30:03.300070+0000","last_became_active":"2024-04-21T10:29:00.923100+0000","last_became_peered":"2024-04-21T10:29:00.923100+0000","last_unstale":"2024-04-21T10:30:03.300070+0000","last_undegraded":"2024-04-21T10:30:03.300070+0000","last_fullsized":"2024-04-21T10:30:03.300070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:45.692961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307802+0000","last_change":"2024-04-21T10:29:01.938131+0000","last_active":"2024-04-21T10:30:03.307802+0000","last_peered":"2024-04-21T10:30:03.307802+0000","last_clean":"2024-04-21T10:30:03.307802+0000","last_became_active":"2024-04-21T10:29:01.937752+0000","last_became_peered":"2024-04-21T10:29:01.937752+0000","last_unstale":"2024-04-21T10:30:03.307802+0000","last_undegraded":"2024-04-21T10:30:03.307802+0000","last_fullsized":"2024-04-21T10:30:03.307802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:06:49.383554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307979+0000","last_change":"2024-04-21T10:29:00.929877+0000","last_active":"2024-04-21T10:30:03.307979+0000","last_peered":"2024-04-21T10:30:03.307979+0000","last_clean":"2024-04-21T10:30:03.307979+0000","last_became_active":"2024-04-21T10:29:00.929564+0000","last_became_peered":"2024-04-21T10:29:00.929564+0000","last_unstale":"2024-04-21T10:30:03.307979+0000","last_undegraded":"2024-04-21T10:30:03.307979+0000","last_fullsized":"2024-04-21T10:30:03.307979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:35.617533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308242+0000","last_change":"2024-04-21T10:29:01.942782+0000","last_active":"2024-04-21T10:30:03.308242+0000","last_peered":"2024-04-21T10:30:03.308242+0000","last_clean":"2024-04-21T10:30:03.308242+0000","last_became_active":"2024-04-21T10:29:01.940501+0000","last_became_peered":"2024-04-21T10:29:01.940501+0000","last_unstale":"2024-04-21T10:30:03.308242+0000","last_undegraded":"2024-04-21T10:30:03.308242+0000","last_fullsized":"2024-04-21T10:30:03.308242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:04:03.431016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302544+0000","last_change":"2024-04-21T10:29:00.924395+0000","last_active":"2024-04-21T10:30:03.302544+0000","last_peered":"2024-04-21T10:30:03.302544+0000","last_clean":"2024-04-21T10:30:03.302544+0000","last_became_active":"2024-04-21T10:29:00.924045+0000","last_became_peered":"2024-04-21T10:29:00.924045+0000","last_unstale":"2024-04-21T10:30:03.302544+0000","last_undegraded":"2024-04-21T10:30:03.302544+0000","last_fullsized":"2024-04-21T10:30:03.302544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:08:53.922213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518063+0000","last_change":"2024-04-21T10:29:01.934508+0000","last_active":"2024-04-21T10:30:03.518063+0000","last_peered":"2024-04-21T10:30:03.518063+0000","last_clean":"2024-04-21T10:30:03.518063+0000","last_became_active":"2024-04-21T10:29:01.934051+0000","last_became_peered":"2024-04-21T10:29:01.934051+0000","last_unstale":"2024-04-21T10:30:03.518063+0000","last_undegraded":"2024-04-21T10:30:03.518063+0000","last_fullsized":"2024-04-21T10:30:03.518063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:49:27.686603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302573+0000","last_change":"2024-04-21T10:29:00.925850+0000","last_active":"2024-04-21T10:30:03.302573+0000","last_peered":"2024-04-21T10:30:03.302573+0000","last_clean":"2024-04-21T10:30:03.302573+0000","last_became_active":"2024-04-21T10:29:00.925521+0000","last_became_peered":"2024-04-21T10:29:00.925521+0000","last_unstale":"2024-04-21T10:30:03.302573+0000","last_undegraded":"2024-04-21T10:30:03.302573+0000","last_fullsized":"2024-04-21T10:30:03.302573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:36:55.403413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305716+0000","last_change":"2024-04-21T10:29:01.930046+0000","last_active":"2024-04-21T10:30:03.305716+0000","last_peered":"2024-04-21T10:30:03.305716+0000","last_clean":"2024-04-21T10:30:03.305716+0000","last_became_active":"2024-04-21T10:29:01.929736+0000","last_became_peered":"2024-04-21T10:29:01.929736+0000","last_unstale":"2024-04-21T10:30:03.305716+0000","last_undegraded":"2024-04-21T10:30:03.305716+0000","last_fullsized":"2024-04-21T10:30:03.305716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:11:10.833267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:10.910326+0000","last_change":"2024-04-21T10:29:00.927936+0000","last_active":"2024-04-21T10:30:10.910326+0000","last_peered":"2024-04-21T10:30:10.910326+0000","last_clean":"2024-04-21T10:30:10.910326+0000","last_became_active":"2024-04-21T10:29:00.927352+0000","last_became_peered":"2024-04-21T10:29:00.927352+0000","last_unstale":"2024-04-21T10:30:10.910326+0000","last_undegraded":"2024-04-21T10:30:10.910326+0000","last_fullsized":"2024-04-21T10:30:10.910326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:14:10.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.306538+0000","last_change":"2024-04-21T10:30:03.306538+0000","last_active":"2024-04-21T10:30:03.306538+0000","last_peered":"2024-04-21T10:30:03.306538+0000","last_clean":"2024-04-21T10:30:03.306538+0000","last_became_active":"2024-04-21T10:30:03.306197+0000","last_became_peered":"2024-04-21T10:30:03.306197+0000","last_unstale":"2024-04-21T10:30:03.306538+0000","last_undegraded":"2024-04-21T10:30:03.306538+0000","last_fullsized":"2024-04-21T10:30:03.306538+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:28:25.688821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.522106+0000","last_change":"2024-04-21T10:30:03.522106+0000","last_active":"2024-04-21T10:30:03.522106+0000","last_peered":"2024-04-21T10:30:03.522106+0000","last_clean":"2024-04-21T10:30:03.522106+0000","last_became_active":"2024-04-21T10:30:03.519189+0000","last_became_peered":"2024-04-21T10:30:03.519189+0000","last_unstale":"2024-04-21T10:30:03.522106+0000","last_undegraded":"2024-04-21T10:30:03.522106+0000","last_fullsized":"2024-04-21T10:30:03.522106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:49:39.930818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305558+0000","last_change":"2024-04-21T10:29:01.918627+0000","last_active":"2024-04-21T10:30:03.305558+0000","last_peered":"2024-04-21T10:30:03.305558+0000","last_clean":"2024-04-21T10:30:03.305558+0000","last_became_active":"2024-04-21T10:29:01.917993+0000","last_became_peered":"2024-04-21T10:29:01.917993+0000","last_unstale":"2024-04-21T10:30:03.305558+0000","last_undegraded":"2024-04-21T10:30:03.305558+0000","last_fullsized":"2024-04-21T10:30:03.305558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:58.891918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303712+0000","last_change":"2024-04-21T10:29:00.923613+0000","last_active":"2024-04-21T10:30:03.303712+0000","last_peered":"2024-04-21T10:30:03.303712+0000","last_clean":"2024-04-21T10:30:03.303712+0000","last_became_active":"2024-04-21T10:29:00.923001+0000","last_became_peered":"2024-04-21T10:29:00.923001+0000","last_unstale":"2024-04-21T10:30:03.303712+0000","last_undegraded":"2024-04-21T10:30:03.303712+0000","last_fullsized":"2024-04-21T10:30:03.303712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:44:53.593903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307316+0000","last_change":"2024-04-21T10:29:01.934974+0000","last_active":"2024-04-21T10:30:03.307316+0000","last_peered":"2024-04-21T10:30:03.307316+0000","last_clean":"2024-04-21T10:30:03.307316+0000","last_became_active":"2024-04-21T10:29:01.932443+0000","last_became_peered":"2024-04-21T10:29:01.932443+0000","last_unstale":"2024-04-21T10:30:03.307316+0000","last_undegraded":"2024-04-21T10:30:03.307316+0000","last_fullsized":"2024-04-21T10:30:03.307316+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:24:47.190142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305205+0000","last_change":"2024-04-21T10:29:00.928179+0000","last_active":"2024-04-21T10:30:03.305205+0000","last_peered":"2024-04-21T10:30:03.305205+0000","last_clean":"2024-04-21T10:30:03.305205+0000","last_became_active":"2024-04-21T10:29:00.927588+0000","last_became_peered":"2024-04-21T10:29:00.927588+0000","last_unstale":"2024-04-21T10:30:03.305205+0000","last_undegraded":"2024-04-21T10:30:03.305205+0000","last_fullsized":"2024-04-21T10:30:03.305205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:04:41.081424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.299988+0000","last_change":"2024-04-21T10:29:01.944421+0000","last_active":"2024-04-21T10:30:03.299988+0000","last_peered":"2024-04-21T10:30:03.299988+0000","last_clean":"2024-04-21T10:30:03.299988+0000","last_became_active":"2024-04-21T10:29:01.943572+0000","last_became_peered":"2024-04-21T10:29:01.943572+0000","last_unstale":"2024-04-21T10:30:03.299988+0000","last_undegraded":"2024-04-21T10:30:03.299988+0000","last_fullsized":"2024-04-21T10:30:03.299988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:34:01.690600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:10.909801+0000","last_change":"2024-04-21T10:29:00.924406+0000","last_active":"2024-04-21T10:30:10.909801+0000","last_peered":"2024-04-21T10:30:10.909801+0000","last_clean":"2024-04-21T10:30:10.909801+0000","last_became_active":"2024-04-21T10:29:00.924094+0000","last_became_peered":"2024-04-21T10:29:00.924094+0000","last_unstale":"2024-04-21T10:30:10.909801+0000","last_undegraded":"2024-04-21T10:30:10.909801+0000","last_fullsized":"2024-04-21T10:30:10.909801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:44.673751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518659+0000","last_change":"2024-04-21T10:29:01.933048+0000","last_active":"2024-04-21T10:30:03.518659+0000","last_peered":"2024-04-21T10:30:03.518659+0000","last_clean":"2024-04-21T10:30:03.518659+0000","last_became_active":"2024-04-21T10:29:01.932659+0000","last_became_peered":"2024-04-21T10:29:01.932659+0000","last_unstale":"2024-04-21T10:30:03.518659+0000","last_undegraded":"2024-04-21T10:30:03.518659+0000","last_fullsized":"2024-04-21T10:30:03.518659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:22:13.712112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301009+0000","last_change":"2024-04-21T10:29:00.923020+0000","last_active":"2024-04-21T10:30:03.301009+0000","last_peered":"2024-04-21T10:30:03.301009+0000","last_clean":"2024-04-21T10:30:03.301009+0000","last_became_active":"2024-04-21T10:29:00.922678+0000","last_became_peered":"2024-04-21T10:29:00.922678+0000","last_unstale":"2024-04-21T10:30:03.301009+0000","last_undegraded":"2024-04-21T10:30:03.301009+0000","last_fullsized":"2024-04-21T10:30:03.301009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:13:34.698082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308326+0000","last_change":"2024-04-21T10:29:01.947833+0000","last_active":"2024-04-21T10:30:03.308326+0000","last_peered":"2024-04-21T10:30:03.308326+0000","last_clean":"2024-04-21T10:30:03.308326+0000","last_became_active":"2024-04-21T10:29:01.947351+0000","last_became_peered":"2024-04-21T10:29:01.947351+0000","last_unstale":"2024-04-21T10:30:03.308326+0000","last_undegraded":"2024-04-21T10:30:03.308326+0000","last_fullsized":"2024-04-21T10:30:03.308326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:54.538731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305809+0000","last_change":"2024-04-21T10:29:00.932309+0000","last_active":"2024-04-21T10:30:03.305809+0000","last_peered":"2024-04-21T10:30:03.305809+0000","last_clean":"2024-04-21T10:30:03.305809+0000","last_became_active":"2024-04-21T10:29:00.931935+0000","last_became_peered":"2024-04-21T10:29:00.931935+0000","last_unstale":"2024-04-21T10:30:03.305809+0000","last_undegraded":"2024-04-21T10:30:03.305809+0000","last_fullsized":"2024-04-21T10:30:03.305809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:37:37.604381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518174+0000","last_change":"2024-04-21T10:29:01.946543+0000","last_active":"2024-04-21T10:30:03.518174+0000","last_peered":"2024-04-21T10:30:03.518174+0000","last_clean":"2024-04-21T10:30:03.518174+0000","last_became_active":"2024-04-21T10:29:01.946210+0000","last_became_peered":"2024-04-21T10:29:01.946210+0000","last_unstale":"2024-04-21T10:30:03.518174+0000","last_undegraded":"2024-04-21T10:30:03.518174+0000","last_fullsized":"2024-04-21T10:30:03.518174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:49.368851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'30","reported_seq":192,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:11.922141+0000","last_change":"2024-04-21T10:29:00.924109+0000","last_active":"2024-04-21T10:30:11.922141+0000","last_peered":"2024-04-21T10:30:11.922141+0000","last_clean":"2024-04-21T10:30:11.922141+0000","last_became_active":"2024-04-21T10:29:00.923735+0000","last_became_peered":"2024-04-21T10:29:00.923735+0000","last_unstale":"2024-04-21T10:30:11.922141+0000","last_undegraded":"2024-04-21T10:30:11.922141+0000","last_fullsized":"2024-04-21T10:30:11.922141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:13:54.015774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":143,"num_read_kb":130,"num_write":31,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302268+0000","last_change":"2024-04-21T10:29:01.937211+0000","last_active":"2024-04-21T10:30:03.302268+0000","last_peered":"2024-04-21T10:30:03.302268+0000","last_clean":"2024-04-21T10:30:03.302268+0000","last_became_active":"2024-04-21T10:29:01.936891+0000","last_became_peered":"2024-04-21T10:29:01.936891+0000","last_unstale":"2024-04-21T10:30:03.302268+0000","last_undegraded":"2024-04-21T10:30:03.302268+0000","last_fullsized":"2024-04-21T10:30:03.302268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:26.062236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301541+0000","last_change":"2024-04-21T10:29:00.920626+0000","last_active":"2024-04-21T10:30:03.301541+0000","last_peered":"2024-04-21T10:30:03.301541+0000","last_clean":"2024-04-21T10:30:03.301541+0000","last_became_active":"2024-04-21T10:29:00.920317+0000","last_became_peered":"2024-04-21T10:29:00.920317+0000","last_unstale":"2024-04-21T10:30:03.301541+0000","last_undegraded":"2024-04-21T10:30:03.301541+0000","last_fullsized":"2024-04-21T10:30:03.301541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:41:18.373403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303859+0000","last_change":"2024-04-21T10:29:01.942494+0000","last_active":"2024-04-21T10:30:03.303859+0000","last_peered":"2024-04-21T10:30:03.303859+0000","last_clean":"2024-04-21T10:30:03.303859+0000","last_became_active":"2024-04-21T10:29:01.942151+0000","last_became_peered":"2024-04-21T10:29:01.942151+0000","last_unstale":"2024-04-21T10:30:03.303859+0000","last_undegraded":"2024-04-21T10:30:03.303859+0000","last_fullsized":"2024-04-21T10:30:03.303859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:39.071158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305052+0000","last_change":"2024-04-21T10:29:00.928221+0000","last_active":"2024-04-21T10:30:03.305052+0000","last_peered":"2024-04-21T10:30:03.305052+0000","last_clean":"2024-04-21T10:30:03.305052+0000","last_became_active":"2024-04-21T10:29:00.927873+0000","last_became_peered":"2024-04-21T10:29:00.927873+0000","last_unstale":"2024-04-21T10:30:03.305052+0000","last_undegraded":"2024-04-21T10:30:03.305052+0000","last_fullsized":"2024-04-21T10:30:03.305052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:14.898456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301059+0000","last_change":"2024-04-21T10:29:01.930375+0000","last_active":"2024-04-21T10:30:03.301059+0000","last_peered":"2024-04-21T10:30:03.301059+0000","last_clean":"2024-04-21T10:30:03.301059+0000","last_became_active":"2024-04-21T10:29:01.930009+0000","last_became_peered":"2024-04-21T10:29:01.930009+0000","last_unstale":"2024-04-21T10:30:03.301059+0000","last_undegraded":"2024-04-21T10:30:03.301059+0000","last_fullsized":"2024-04-21T10:30:03.301059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:34.111797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:10.910233+0000","last_change":"2024-04-21T10:29:00.922981+0000","last_active":"2024-04-21T10:30:10.910233+0000","last_peered":"2024-04-21T10:30:10.910233+0000","last_clean":"2024-04-21T10:30:10.910233+0000","last_became_active":"2024-04-21T10:29:00.922668+0000","last_became_peered":"2024-04-21T10:29:00.922668+0000","last_unstale":"2024-04-21T10:30:10.910233+0000","last_undegraded":"2024-04-21T10:30:10.910233+0000","last_fullsized":"2024-04-21T10:30:10.910233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:47.109375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300763+0000","last_change":"2024-04-21T10:29:01.942530+0000","last_active":"2024-04-21T10:30:03.300763+0000","last_peered":"2024-04-21T10:30:03.300763+0000","last_clean":"2024-04-21T10:30:03.300763+0000","last_became_active":"2024-04-21T10:29:01.942162+0000","last_became_peered":"2024-04-21T10:29:01.942162+0000","last_unstale":"2024-04-21T10:30:03.300763+0000","last_undegraded":"2024-04-21T10:30:03.300763+0000","last_fullsized":"2024-04-21T10:30:03.300763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:46:19.138930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.544712+0000","last_change":"2024-04-21T10:30:03.544712+0000","last_active":"2024-04-21T10:30:03.544712+0000","last_peered":"2024-04-21T10:30:03.544712+0000","last_clean":"2024-04-21T10:30:03.544712+0000","last_became_active":"2024-04-21T10:30:03.519730+0000","last_became_peered":"2024-04-21T10:30:03.519730+0000","last_unstale":"2024-04-21T10:30:03.544712+0000","last_undegraded":"2024-04-21T10:30:03.544712+0000","last_fullsized":"2024-04-21T10:30:03.544712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:54:33.184653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302388+0000","last_change":"2024-04-21T10:29:01.929696+0000","last_active":"2024-04-21T10:30:03.302388+0000","last_peered":"2024-04-21T10:30:03.302388+0000","last_clean":"2024-04-21T10:30:03.302388+0000","last_became_active":"2024-04-21T10:29:01.929328+0000","last_became_peered":"2024-04-21T10:29:01.929328+0000","last_unstale":"2024-04-21T10:30:03.302388+0000","last_undegraded":"2024-04-21T10:30:03.302388+0000","last_fullsized":"2024-04-21T10:30:03.302388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:55:42.735595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302375+0000","last_change":"2024-04-21T10:29:00.925981+0000","last_active":"2024-04-21T10:30:03.302375+0000","last_peered":"2024-04-21T10:30:03.302375+0000","last_clean":"2024-04-21T10:30:03.302375+0000","last_became_active":"2024-04-21T10:29:00.925560+0000","last_became_peered":"2024-04-21T10:29:00.925560+0000","last_unstale":"2024-04-21T10:30:03.302375+0000","last_undegraded":"2024-04-21T10:30:03.302375+0000","last_fullsized":"2024-04-21T10:30:03.302375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:37.467522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303410+0000","last_change":"2024-04-21T10:29:01.941961+0000","last_active":"2024-04-21T10:30:03.303410+0000","last_peered":"2024-04-21T10:30:03.303410+0000","last_clean":"2024-04-21T10:30:03.303410+0000","last_became_active":"2024-04-21T10:29:01.941649+0000","last_became_peered":"2024-04-21T10:29:01.941649+0000","last_unstale":"2024-04-21T10:30:03.303410+0000","last_undegraded":"2024-04-21T10:30:03.303410+0000","last_fullsized":"2024-04-21T10:30:03.303410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:00:56.803856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.299991+0000","last_change":"2024-04-21T10:29:00.929504+0000","last_active":"2024-04-21T10:30:03.299991+0000","last_peered":"2024-04-21T10:30:03.299991+0000","last_clean":"2024-04-21T10:30:03.299991+0000","last_became_active":"2024-04-21T10:29:00.929172+0000","last_became_peered":"2024-04-21T10:29:00.929172+0000","last_unstale":"2024-04-21T10:30:03.299991+0000","last_undegraded":"2024-04-21T10:30:03.299991+0000","last_fullsized":"2024-04-21T10:30:03.299991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:40.512175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303818+0000","last_change":"2024-04-21T10:29:01.939333+0000","last_active":"2024-04-21T10:30:03.303818+0000","last_peered":"2024-04-21T10:30:03.303818+0000","last_clean":"2024-04-21T10:30:03.303818+0000","last_became_active":"2024-04-21T10:29:01.938924+0000","last_became_peered":"2024-04-21T10:29:01.938924+0000","last_unstale":"2024-04-21T10:30:03.303818+0000","last_undegraded":"2024-04-21T10:30:03.303818+0000","last_fullsized":"2024-04-21T10:30:03.303818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:20:01.287604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300883+0000","last_change":"2024-04-21T10:29:00.927455+0000","last_active":"2024-04-21T10:30:03.300883+0000","last_peered":"2024-04-21T10:30:03.300883+0000","last_clean":"2024-04-21T10:30:03.300883+0000","last_became_active":"2024-04-21T10:29:00.927180+0000","last_became_peered":"2024-04-21T10:29:00.927180+0000","last_unstale":"2024-04-21T10:30:03.300883+0000","last_undegraded":"2024-04-21T10:30:03.300883+0000","last_fullsized":"2024-04-21T10:30:03.300883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:11:57.166607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303587+0000","last_change":"2024-04-21T10:29:01.937646+0000","last_active":"2024-04-21T10:30:03.303587+0000","last_peered":"2024-04-21T10:30:03.303587+0000","last_clean":"2024-04-21T10:30:03.303587+0000","last_became_active":"2024-04-21T10:29:01.937216+0000","last_became_peered":"2024-04-21T10:29:01.937216+0000","last_unstale":"2024-04-21T10:30:03.303587+0000","last_undegraded":"2024-04-21T10:30:03.303587+0000","last_fullsized":"2024-04-21T10:30:03.303587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:38.265252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305317+0000","last_change":"2024-04-21T10:29:01.926860+0000","last_active":"2024-04-21T10:30:03.305317+0000","last_peered":"2024-04-21T10:30:03.305317+0000","last_clean":"2024-04-21T10:30:03.305317+0000","last_became_active":"2024-04-21T10:29:01.925951+0000","last_became_peered":"2024-04-21T10:29:01.925951+0000","last_unstale":"2024-04-21T10:30:03.305317+0000","last_undegraded":"2024-04-21T10:30:03.305317+0000","last_fullsized":"2024-04-21T10:30:03.305317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:01:55.127301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:09.908483+0000","last_change":"2024-04-21T10:29:00.926502+0000","last_active":"2024-04-21T10:30:09.908483+0000","last_peered":"2024-04-21T10:30:09.908483+0000","last_clean":"2024-04-21T10:30:09.908483+0000","last_became_active":"2024-04-21T10:29:00.926175+0000","last_became_peered":"2024-04-21T10:29:00.926175+0000","last_unstale":"2024-04-21T10:30:09.908483+0000","last_undegraded":"2024-04-21T10:30:09.908483+0000","last_fullsized":"2024-04-21T10:30:09.908483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:40:34.774085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.520292+0000","last_change":"2024-04-21T10:30:03.519801+0000","last_active":"2024-04-21T10:30:03.520292+0000","last_peered":"2024-04-21T10:30:03.520292+0000","last_clean":"2024-04-21T10:30:03.520292+0000","last_became_active":"2024-04-21T10:30:03.519351+0000","last_became_peered":"2024-04-21T10:30:03.519351+0000","last_unstale":"2024-04-21T10:30:03.520292+0000","last_undegraded":"2024-04-21T10:30:03.520292+0000","last_fullsized":"2024-04-21T10:30:03.520292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:13:36.675062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518520+0000","last_change":"2024-04-21T10:29:00.921748+0000","last_active":"2024-04-21T10:30:03.518520+0000","last_peered":"2024-04-21T10:30:03.518520+0000","last_clean":"2024-04-21T10:30:03.518520+0000","last_became_active":"2024-04-21T10:29:00.921408+0000","last_became_peered":"2024-04-21T10:29:00.921408+0000","last_unstale":"2024-04-21T10:30:03.518520+0000","last_undegraded":"2024-04-21T10:30:03.518520+0000","last_fullsized":"2024-04-21T10:30:03.518520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:52:27.606280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300849+0000","last_change":"2024-04-21T10:27:49.787009+0000","last_active":"2024-04-21T10:30:03.300849+0000","last_peered":"2024-04-21T10:30:03.300849+0000","last_clean":"2024-04-21T10:30:03.300849+0000","last_became_active":"2024-04-21T10:27:49.786530+0000","last_became_peered":"2024-04-21T10:27:49.786530+0000","last_unstale":"2024-04-21T10:30:03.300849+0000","last_undegraded":"2024-04-21T10:30:03.300849+0000","last_fullsized":"2024-04-21T10:30:03.300849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_clean_scrub_stamp":"2024-04-21T10:24:28.367927+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:20:12.106231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305720+0000","last_change":"2024-04-21T10:30:03.305720+0000","last_active":"2024-04-21T10:30:03.305720+0000","last_peered":"2024-04-21T10:30:03.305720+0000","last_clean":"2024-04-21T10:30:03.305720+0000","last_became_active":"2024-04-21T10:30:03.305368+0000","last_became_peered":"2024-04-21T10:30:03.305368+0000","last_unstale":"2024-04-21T10:30:03.305720+0000","last_undegraded":"2024-04-21T10:30:03.305720+0000","last_fullsized":"2024-04-21T10:30:03.305720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:16.668743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300951+0000","last_change":"2024-04-21T10:29:00.927521+0000","last_active":"2024-04-21T10:30:03.300951+0000","last_peered":"2024-04-21T10:30:03.300951+0000","last_clean":"2024-04-21T10:30:03.300951+0000","last_became_active":"2024-04-21T10:29:00.927202+0000","last_became_peered":"2024-04-21T10:29:00.927202+0000","last_unstale":"2024-04-21T10:30:03.300951+0000","last_undegraded":"2024-04-21T10:30:03.300951+0000","last_fullsized":"2024-04-21T10:30:03.300951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:11:32.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302357+0000","last_change":"2024-04-21T10:29:00.930728+0000","last_active":"2024-04-21T10:30:03.302357+0000","last_peered":"2024-04-21T10:30:03.302357+0000","last_clean":"2024-04-21T10:30:03.302357+0000","last_became_active":"2024-04-21T10:29:00.930457+0000","last_became_peered":"2024-04-21T10:29:00.930457+0000","last_unstale":"2024-04-21T10:30:03.302357+0000","last_undegraded":"2024-04-21T10:30:03.302357+0000","last_fullsized":"2024-04-21T10:30:03.302357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:28.032954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.304625+0000","last_change":"2024-04-21T10:29:01.932278+0000","last_active":"2024-04-21T10:30:03.304625+0000","last_peered":"2024-04-21T10:30:03.304625+0000","last_clean":"2024-04-21T10:30:03.304625+0000","last_became_active":"2024-04-21T10:29:01.931774+0000","last_became_peered":"2024-04-21T10:29:01.931774+0000","last_unstale":"2024-04-21T10:30:03.304625+0000","last_undegraded":"2024-04-21T10:30:03.304625+0000","last_fullsized":"2024-04-21T10:30:03.304625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:34:25.858475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.300194+0000","last_change":"2024-04-21T10:29:00.926071+0000","last_active":"2024-04-21T10:30:03.300194+0000","last_peered":"2024-04-21T10:30:03.300194+0000","last_clean":"2024-04-21T10:30:03.300194+0000","last_became_active":"2024-04-21T10:29:00.925607+0000","last_became_peered":"2024-04-21T10:29:00.925607+0000","last_unstale":"2024-04-21T10:30:03.300194+0000","last_undegraded":"2024-04-21T10:30:03.300194+0000","last_fullsized":"2024-04-21T10:30:03.300194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:26.350652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308249+0000","last_change":"2024-04-21T10:29:01.941365+0000","last_active":"2024-04-21T10:30:03.308249+0000","last_peered":"2024-04-21T10:30:03.308249+0000","last_clean":"2024-04-21T10:30:03.308249+0000","last_became_active":"2024-04-21T10:29:01.940102+0000","last_became_peered":"2024-04-21T10:29:01.940102+0000","last_unstale":"2024-04-21T10:30:03.308249+0000","last_undegraded":"2024-04-21T10:30:03.308249+0000","last_fullsized":"2024-04-21T10:30:03.308249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:21.976855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.519407+0000","last_change":"2024-04-21T10:29:00.924322+0000","last_active":"2024-04-21T10:30:03.519407+0000","last_peered":"2024-04-21T10:30:03.519407+0000","last_clean":"2024-04-21T10:30:03.519407+0000","last_became_active":"2024-04-21T10:29:00.924016+0000","last_became_peered":"2024-04-21T10:29:00.924016+0000","last_unstale":"2024-04-21T10:30:03.519407+0000","last_undegraded":"2024-04-21T10:30:03.519407+0000","last_fullsized":"2024-04-21T10:30:03.519407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:49:24.004951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308135+0000","last_change":"2024-04-21T10:29:01.946315+0000","last_active":"2024-04-21T10:30:03.308135+0000","last_peered":"2024-04-21T10:30:03.308135+0000","last_clean":"2024-04-21T10:30:03.308135+0000","last_became_active":"2024-04-21T10:29:01.945920+0000","last_became_peered":"2024-04-21T10:29:01.945920+0000","last_unstale":"2024-04-21T10:30:03.308135+0000","last_undegraded":"2024-04-21T10:30:03.308135+0000","last_fullsized":"2024-04-21T10:30:03.308135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:10:52.361615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518803+0000","last_change":"2024-04-21T10:29:00.919086+0000","last_active":"2024-04-21T10:30:03.518803+0000","last_peered":"2024-04-21T10:30:03.518803+0000","last_clean":"2024-04-21T10:30:03.518803+0000","last_became_active":"2024-04-21T10:29:00.918727+0000","last_became_peered":"2024-04-21T10:29:00.918727+0000","last_unstale":"2024-04-21T10:30:03.518803+0000","last_undegraded":"2024-04-21T10:30:03.518803+0000","last_fullsized":"2024-04-21T10:30:03.518803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:53:27.209135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301802+0000","last_change":"2024-04-21T10:29:01.944037+0000","last_active":"2024-04-21T10:30:03.301802+0000","last_peered":"2024-04-21T10:30:03.301802+0000","last_clean":"2024-04-21T10:30:03.301802+0000","last_became_active":"2024-04-21T10:29:01.943692+0000","last_became_peered":"2024-04-21T10:29:01.943692+0000","last_unstale":"2024-04-21T10:30:03.301802+0000","last_undegraded":"2024-04-21T10:30:03.301802+0000","last_fullsized":"2024-04-21T10:30:03.301802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:51.285535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307485+0000","last_change":"2024-04-21T10:29:00.930937+0000","last_active":"2024-04-21T10:30:03.307485+0000","last_peered":"2024-04-21T10:30:03.307485+0000","last_clean":"2024-04-21T10:30:03.307485+0000","last_became_active":"2024-04-21T10:29:00.930348+0000","last_became_peered":"2024-04-21T10:29:00.930348+0000","last_unstale":"2024-04-21T10:30:03.307485+0000","last_undegraded":"2024-04-21T10:30:03.307485+0000","last_fullsized":"2024-04-21T10:30:03.307485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:43:33.801432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.304513+0000","last_change":"2024-04-21T10:29:01.937116+0000","last_active":"2024-04-21T10:30:03.304513+0000","last_peered":"2024-04-21T10:30:03.304513+0000","last_clean":"2024-04-21T10:30:03.304513+0000","last_became_active":"2024-04-21T10:29:01.936756+0000","last_became_peered":"2024-04-21T10:29:01.936756+0000","last_unstale":"2024-04-21T10:30:03.304513+0000","last_undegraded":"2024-04-21T10:30:03.304513+0000","last_fullsized":"2024-04-21T10:30:03.304513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:02.732399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.312469+0000","last_change":"2024-04-21T10:30:03.312469+0000","last_active":"2024-04-21T10:30:03.312469+0000","last_peered":"2024-04-21T10:30:03.312469+0000","last_clean":"2024-04-21T10:30:03.312469+0000","last_became_active":"2024-04-21T10:30:03.310027+0000","last_became_peered":"2024-04-21T10:30:03.310027+0000","last_unstale":"2024-04-21T10:30:03.312469+0000","last_undegraded":"2024-04-21T10:30:03.312469+0000","last_fullsized":"2024-04-21T10:30:03.312469+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:12:18.582539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308475+0000","last_change":"2024-04-21T10:29:01.940677+0000","last_active":"2024-04-21T10:30:03.308475+0000","last_peered":"2024-04-21T10:30:03.308475+0000","last_clean":"2024-04-21T10:30:03.308475+0000","last_became_active":"2024-04-21T10:29:01.940296+0000","last_became_peered":"2024-04-21T10:29:01.940296+0000","last_unstale":"2024-04-21T10:30:03.308475+0000","last_undegraded":"2024-04-21T10:30:03.308475+0000","last_fullsized":"2024-04-21T10:30:03.308475+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:24:52.662041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302074+0000","last_change":"2024-04-21T10:29:00.924392+0000","last_active":"2024-04-21T10:30:03.302074+0000","last_peered":"2024-04-21T10:30:03.302074+0000","last_clean":"2024-04-21T10:30:03.302074+0000","last_became_active":"2024-04-21T10:29:00.924029+0000","last_became_peered":"2024-04-21T10:29:00.924029+0000","last_unstale":"2024-04-21T10:30:03.302074+0000","last_undegraded":"2024-04-21T10:30:03.302074+0000","last_fullsized":"2024-04-21T10:30:03.302074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:23:50.726978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301081+0000","last_change":"2024-04-21T10:29:01.939687+0000","last_active":"2024-04-21T10:30:03.301081+0000","last_peered":"2024-04-21T10:30:03.301081+0000","last_clean":"2024-04-21T10:30:03.301081+0000","last_became_active":"2024-04-21T10:29:01.939343+0000","last_became_peered":"2024-04-21T10:29:01.939343+0000","last_unstale":"2024-04-21T10:30:03.301081+0000","last_undegraded":"2024-04-21T10:30:03.301081+0000","last_fullsized":"2024-04-21T10:30:03.301081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:50.481998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301786+0000","last_change":"2024-04-21T10:29:00.923115+0000","last_active":"2024-04-21T10:30:03.301786+0000","last_peered":"2024-04-21T10:30:03.301786+0000","last_clean":"2024-04-21T10:30:03.301786+0000","last_became_active":"2024-04-21T10:29:00.922707+0000","last_became_peered":"2024-04-21T10:29:00.922707+0000","last_unstale":"2024-04-21T10:30:03.301786+0000","last_undegraded":"2024-04-21T10:30:03.301786+0000","last_fullsized":"2024-04-21T10:30:03.301786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:55:53.419904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301150+0000","last_change":"2024-04-21T10:29:01.940666+0000","last_active":"2024-04-21T10:30:03.301150+0000","last_peered":"2024-04-21T10:30:03.301150+0000","last_clean":"2024-04-21T10:30:03.301150+0000","last_became_active":"2024-04-21T10:29:01.940331+0000","last_became_peered":"2024-04-21T10:29:01.940331+0000","last_unstale":"2024-04-21T10:30:03.301150+0000","last_undegraded":"2024-04-21T10:30:03.301150+0000","last_fullsized":"2024-04-21T10:30:03.301150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:52:41.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.305749+0000","last_change":"2024-04-21T10:29:00.927873+0000","last_active":"2024-04-21T10:30:03.305749+0000","last_peered":"2024-04-21T10:30:03.305749+0000","last_clean":"2024-04-21T10:30:03.305749+0000","last_became_active":"2024-04-21T10:29:00.927403+0000","last_became_peered":"2024-04-21T10:29:00.927403+0000","last_unstale":"2024-04-21T10:30:03.305749+0000","last_undegraded":"2024-04-21T10:30:03.305749+0000","last_fullsized":"2024-04-21T10:30:03.305749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:05:05.276978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301019+0000","last_change":"2024-04-21T10:29:01.944725+0000","last_active":"2024-04-21T10:30:03.301019+0000","last_peered":"2024-04-21T10:30:03.301019+0000","last_clean":"2024-04-21T10:30:03.301019+0000","last_became_active":"2024-04-21T10:29:01.944326+0000","last_became_peered":"2024-04-21T10:29:01.944326+0000","last_unstale":"2024-04-21T10:30:03.301019+0000","last_undegraded":"2024-04-21T10:30:03.301019+0000","last_fullsized":"2024-04-21T10:30:03.301019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:50.209064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303976+0000","last_change":"2024-04-21T10:29:00.924694+0000","last_active":"2024-04-21T10:30:03.303976+0000","last_peered":"2024-04-21T10:30:03.303976+0000","last_clean":"2024-04-21T10:30:03.303976+0000","last_became_active":"2024-04-21T10:29:00.924373+0000","last_became_peered":"2024-04-21T10:29:00.924373+0000","last_unstale":"2024-04-21T10:30:03.303976+0000","last_undegraded":"2024-04-21T10:30:03.303976+0000","last_fullsized":"2024-04-21T10:30:03.303976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:10:14.681630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301133+0000","last_change":"2024-04-21T10:29:01.939277+0000","last_active":"2024-04-21T10:30:03.301133+0000","last_peered":"2024-04-21T10:30:03.301133+0000","last_clean":"2024-04-21T10:30:03.301133+0000","last_became_active":"2024-04-21T10:29:01.938844+0000","last_became_peered":"2024-04-21T10:29:01.938844+0000","last_unstale":"2024-04-21T10:30:03.301133+0000","last_undegraded":"2024-04-21T10:30:03.301133+0000","last_fullsized":"2024-04-21T10:30:03.301133+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:38:14.553868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.304616+0000","last_change":"2024-04-21T10:29:01.942654+0000","last_active":"2024-04-21T10:30:03.304616+0000","last_peered":"2024-04-21T10:30:03.304616+0000","last_clean":"2024-04-21T10:30:03.304616+0000","last_became_active":"2024-04-21T10:29:01.942324+0000","last_became_peered":"2024-04-21T10:29:01.942324+0000","last_unstale":"2024-04-21T10:30:03.304616+0000","last_undegraded":"2024-04-21T10:30:03.304616+0000","last_fullsized":"2024-04-21T10:30:03.304616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:19.027138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307328+0000","last_change":"2024-04-21T10:30:03.307328+0000","last_active":"2024-04-21T10:30:03.307328+0000","last_peered":"2024-04-21T10:30:03.307328+0000","last_clean":"2024-04-21T10:30:03.307328+0000","last_became_active":"2024-04-21T10:30:03.306976+0000","last_became_peered":"2024-04-21T10:30:03.306976+0000","last_unstale":"2024-04-21T10:30:03.307328+0000","last_undegraded":"2024-04-21T10:30:03.307328+0000","last_fullsized":"2024-04-21T10:30:03.307328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:19:31.739372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.301467+0000","last_change":"2024-04-21T10:29:01.944405+0000","last_active":"2024-04-21T10:30:03.301467+0000","last_peered":"2024-04-21T10:30:03.301467+0000","last_clean":"2024-04-21T10:30:03.301467+0000","last_became_active":"2024-04-21T10:29:01.944085+0000","last_became_peered":"2024-04-21T10:29:01.944085+0000","last_unstale":"2024-04-21T10:30:03.301467+0000","last_undegraded":"2024-04-21T10:30:03.301467+0000","last_fullsized":"2024-04-21T10:30:03.301467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:13.299859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307414+0000","last_change":"2024-04-21T10:29:01.941054+0000","last_active":"2024-04-21T10:30:03.307414+0000","last_peered":"2024-04-21T10:30:03.307414+0000","last_clean":"2024-04-21T10:30:03.307414+0000","last_became_active":"2024-04-21T10:29:01.940243+0000","last_became_peered":"2024-04-21T10:29:01.940243+0000","last_unstale":"2024-04-21T10:30:03.307414+0000","last_undegraded":"2024-04-21T10:30:03.307414+0000","last_fullsized":"2024-04-21T10:30:03.307414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:31:04.222026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307984+0000","last_change":"2024-04-21T10:30:03.307984+0000","last_active":"2024-04-21T10:30:03.307984+0000","last_peered":"2024-04-21T10:30:03.307984+0000","last_clean":"2024-04-21T10:30:03.307984+0000","last_became_active":"2024-04-21T10:30:03.307584+0000","last_became_peered":"2024-04-21T10:30:03.307584+0000","last_unstale":"2024-04-21T10:30:03.307984+0000","last_undegraded":"2024-04-21T10:30:03.307984+0000","last_fullsized":"2024-04-21T10:30:03.307984+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:12.699665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303423+0000","last_change":"2024-04-21T10:29:01.947268+0000","last_active":"2024-04-21T10:30:03.303423+0000","last_peered":"2024-04-21T10:30:03.303423+0000","last_clean":"2024-04-21T10:30:03.303423+0000","last_became_active":"2024-04-21T10:29:01.946921+0000","last_became_peered":"2024-04-21T10:29:01.946921+0000","last_unstale":"2024-04-21T10:30:03.303423+0000","last_undegraded":"2024-04-21T10:30:03.303423+0000","last_fullsized":"2024-04-21T10:30:03.303423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:50:43.784920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.302177+0000","last_change":"2024-04-21T10:29:01.949969+0000","last_active":"2024-04-21T10:30:03.302177+0000","last_peered":"2024-04-21T10:30:03.302177+0000","last_clean":"2024-04-21T10:30:03.302177+0000","last_became_active":"2024-04-21T10:29:01.949487+0000","last_became_peered":"2024-04-21T10:29:01.949487+0000","last_unstale":"2024-04-21T10:30:03.302177+0000","last_undegraded":"2024-04-21T10:30:03.302177+0000","last_fullsized":"2024-04-21T10:30:03.302177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:26:59.620209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.308151+0000","last_change":"2024-04-21T10:29:01.937904+0000","last_active":"2024-04-21T10:30:03.308151+0000","last_peered":"2024-04-21T10:30:03.308151+0000","last_clean":"2024-04-21T10:30:03.308151+0000","last_became_active":"2024-04-21T10:29:01.937504+0000","last_became_peered":"2024-04-21T10:29:01.937504+0000","last_unstale":"2024-04-21T10:30:03.308151+0000","last_undegraded":"2024-04-21T10:30:03.308151+0000","last_fullsized":"2024-04-21T10:30:03.308151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:58:24.245118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.310042+0000","last_change":"2024-04-21T10:30:03.310042+0000","last_active":"2024-04-21T10:30:03.310042+0000","last_peered":"2024-04-21T10:30:03.310042+0000","last_clean":"2024-04-21T10:30:03.310042+0000","last_became_active":"2024-04-21T10:30:03.309663+0000","last_became_peered":"2024-04-21T10:30:03.309663+0000","last_unstale":"2024-04-21T10:30:03.310042+0000","last_undegraded":"2024-04-21T10:30:03.310042+0000","last_fullsized":"2024-04-21T10:30:03.310042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:19.906949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.307387+0000","last_change":"2024-04-21T10:29:01.936880+0000","last_active":"2024-04-21T10:30:03.307387+0000","last_peered":"2024-04-21T10:30:03.307387+0000","last_clean":"2024-04-21T10:30:03.307387+0000","last_became_active":"2024-04-21T10:29:01.936555+0000","last_became_peered":"2024-04-21T10:29:01.936555+0000","last_unstale":"2024-04-21T10:30:03.307387+0000","last_undegraded":"2024-04-21T10:30:03.307387+0000","last_fullsized":"2024-04-21T10:30:03.307387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:32.831606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.518462+0000","last_change":"2024-04-21T10:29:01.943238+0000","last_active":"2024-04-21T10:30:03.518462+0000","last_peered":"2024-04-21T10:30:03.518462+0000","last_clean":"2024-04-21T10:30:03.518462+0000","last_became_active":"2024-04-21T10:29:01.942841+0000","last_became_peered":"2024-04-21T10:29:01.942841+0000","last_unstale":"2024-04-21T10:30:03.518462+0000","last_undegraded":"2024-04-21T10:30:03.518462+0000","last_fullsized":"2024-04-21T10:30:03.518462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:59:48.551524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303779+0000","last_change":"2024-04-21T10:29:01.933712+0000","last_active":"2024-04-21T10:30:03.303779+0000","last_peered":"2024-04-21T10:30:03.303779+0000","last_clean":"2024-04-21T10:30:03.303779+0000","last_became_active":"2024-04-21T10:29:01.933399+0000","last_became_peered":"2024-04-21T10:29:01.933399+0000","last_unstale":"2024-04-21T10:30:03.303779+0000","last_undegraded":"2024-04-21T10:30:03.303779+0000","last_fullsized":"2024-04-21T10:30:03.303779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:34:16.346403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T10:30:03.303235+0000","last_change":"2024-04-21T10:29:01.947225+0000","last_active":"2024-04-21T10:30:03.303235+0000","last_peered":"2024-04-21T10:30:03.303235+0000","last_clean":"2024-04-21T10:30:03.303235+0000","last_became_active":"2024-04-21T10:29:01.946890+0000","last_became_peered":"2024-04-21T10:29:01.946890+0000","last_unstale":"2024-04-21T10:30:03.303235+0000","last_undegraded":"2024-04-21T10:30:03.303235+0000","last_fullsized":"2024-04-21T10:30:03.303235+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:19:00.960767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64024,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":428,"num_read_kb":281,"num_write":158,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":192072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318301,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":463981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579937,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1115764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":464105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005100,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":463913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266736,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1055955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430261,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":463494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":523422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":464765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":524548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":463805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888128,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":978944,"data_stored":464322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691899,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1593344,"data_stored":1055248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":60135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T10:30:18.691 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-21T10:30:18.691 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-21T10:30:18.691 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-21T10:30:18.691 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph health --format=json 2024-04-21T10:30:18.939 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:19.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:19 smithi026 ceph-mon[27965]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:30:19.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:19 smithi026 ceph-mon[27965]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 9 B/s, 0 objects/s recovering 2024-04-21T10:30:19.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:19 smithi169 ceph-mon[31469]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:30:19.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:19 smithi169 ceph-mon[31469]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 9 B/s, 0 objects/s recovering 2024-04-21T10:30:19.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:19 smithi190 ceph-mon[31423]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:30:19.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:19 smithi190 ceph-mon[31423]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 9 B/s, 0 objects/s recovering 2024-04-21T10:30:20.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.448+0000 7f82cad61700 1 -- 172.21.15.26:0/3599549978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82c4102ac0 msgr2=0x7f82c4102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.448+0000 7f82cad61700 1 --2- 172.21.15.26:0/3599549978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82c4102ac0 0x7f82c4102f40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009990 tx=0x7f82b8009ca0 comp rx=0 tx=0).stop 2024-04-21T10:30:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.448+0000 7f82cad61700 1 -- 172.21.15.26:0/3599549978 shutdown_connections 2024-04-21T10:30:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.448+0000 7f82cad61700 1 --2- 172.21.15.26:0/3599549978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4108a70 0x7f82c4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.448+0000 7f82cad61700 1 --2- 172.21.15.26:0/3599549978 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c4103480 0x7f82c410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.448+0000 7f82cad61700 1 --2- 172.21.15.26:0/3599549978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82c4102ac0 0x7f82c4102f40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.448+0000 7f82cad61700 1 -- 172.21.15.26:0/3599549978 >> 172.21.15.26:0/3599549978 conn(0x7f82c4079bb0 msgr2=0x7f82c4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.448+0000 7f82cad61700 1 -- 172.21.15.26:0/3599549978 shutdown_connections 2024-04-21T10:30:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.448+0000 7f82cad61700 1 -- 172.21.15.26:0/3599549978 wait complete. 2024-04-21T10:30:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.448+0000 7f82cad61700 1 Processor -- start 2024-04-21T10:30:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82cad61700 1 -- start start 2024-04-21T10:30:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82cad61700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82c4102ac0 0x7f82c4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82cad61700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c4103480 0x7f82c41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82cad61700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4108a70 0x7f82c419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82cad61700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4112d00 con 0x7f82c4102ac0 2024-04-21T10:30:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82cad61700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4112b80 con 0x7f82c4108a70 2024-04-21T10:30:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82cad61700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4112e80 con 0x7f82c4103480 2024-04-21T10:30:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82c8afd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82c4102ac0 0x7f82c4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82c8afd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82c4102ac0 0x7f82c4199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47798/0 (socket says 172.21.15.26:47798) 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82c8afd700 1 -- 172.21.15.26:0/3901013351 learned_addr learned my addr 172.21.15.26:0/3901013351 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82c92fe700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4108a70 0x7f82c419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82c3fff700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c4103480 0x7f82c41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82c8afd700 1 -- 172.21.15.26:0/3901013351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c4103480 msgr2=0x7f82c41997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82c8afd700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c4103480 0x7f82c41997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82c8afd700 1 -- 172.21.15.26:0/3901013351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4108a70 msgr2=0x7f82c419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82c8afd700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4108a70 0x7f82c419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82c8afd700 1 -- 172.21.15.26:0/3901013351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c419e280 con 0x7f82c4102ac0 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82c92fe700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4108a70 0x7f82c419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.449+0000 7f82c8afd700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82c4102ac0 0x7f82c4199270 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f82b400b530 tx=0x7f82b400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.450+0000 7f82c1ffb700 1 -- 172.21.15.26:0/3901013351 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4015020 con 0x7f82c4102ac0 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.450+0000 7f82c1ffb700 1 -- 172.21.15.26:0/3901013351 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82b4011ce0 con 0x7f82c4102ac0 2024-04-21T10:30:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.450+0000 7f82c1ffb700 1 -- 172.21.15.26:0/3901013351 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4010d20 con 0x7f82c4102ac0 2024-04-21T10:30:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.450+0000 7f82cad61700 1 -- 172.21.15.26:0/3901013351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c419e570 con 0x7f82c4102ac0 2024-04-21T10:30:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.450+0000 7f82cad61700 1 -- 172.21.15.26:0/3901013351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c41a5df0 con 0x7f82c4102ac0 2024-04-21T10:30:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.451+0000 7f82cad61700 1 -- 172.21.15.26:0/3901013351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c4069730 con 0x7f82c4102ac0 2024-04-21T10:30:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.452+0000 7f82c1ffb700 1 -- 172.21.15.26:0/3901013351 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f82b4018020 con 0x7f82c4102ac0 2024-04-21T10:30:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.452+0000 7f82c1ffb700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f82ac0a3d50 0x7f82ac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.453+0000 7f82c3fff700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f82ac0a3d50 0x7f82ac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.453+0000 7f82c1ffb700 1 -- 172.21.15.26:0/3901013351 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f82b40cfcd0 con 0x7f82c4102ac0 2024-04-21T10:30:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.453+0000 7f82c3fff700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f82ac0a3d50 0x7f82ac0a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f82b8006010 tx=0x7f82b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.457+0000 7f82c1ffb700 1 -- 172.21.15.26:0/3901013351 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82b401f020 con 0x7f82c4102ac0 2024-04-21T10:30:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.717+0000 7f82cad61700 1 -- 172.21.15.26:0/3901013351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f82c402df80 con 0x7f82c4102ac0 2024-04-21T10:30:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.717+0000 7f82c1ffb700 1 -- 172.21.15.26:0/3901013351 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f82b409c5f0 con 0x7f82c4102ac0 2024-04-21T10:30:20.718 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:20.718 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-21T10:30:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.720+0000 7f82cad61700 1 -- 172.21.15.26:0/3901013351 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f82ac0a3d50 msgr2=0x7f82ac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.720+0000 7f82cad61700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f82ac0a3d50 0x7f82ac0a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f82b8006010 tx=0x7f82b801a040 comp rx=0 tx=0).stop 2024-04-21T10:30:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.720+0000 7f82cad61700 1 -- 172.21.15.26:0/3901013351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82c4102ac0 msgr2=0x7f82c4199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.720+0000 7f82cad61700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82c4102ac0 0x7f82c4199270 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f82b400b530 tx=0x7f82b400b8f0 comp rx=0 tx=0).stop 2024-04-21T10:30:20.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.720+0000 7f82cad61700 1 -- 172.21.15.26:0/3901013351 shutdown_connections 2024-04-21T10:30:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.720+0000 7f82cad61700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f82ac0a3d50 0x7f82ac0a6210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.720+0000 7f82cad61700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82c4108a70 0x7f82c419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.720+0000 7f82cad61700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82c4102ac0 0x7f82c4199270 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.720+0000 7f82cad61700 1 --2- 172.21.15.26:0/3901013351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82c4103480 0x7f82c41997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.720+0000 7f82cad61700 1 -- 172.21.15.26:0/3901013351 >> 172.21.15.26:0/3901013351 conn(0x7f82c4079bb0 msgr2=0x7f82c41019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.720+0000 7f82cad61700 1 -- 172.21.15.26:0/3901013351 shutdown_connections 2024-04-21T10:30:20.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:20.720+0000 7f82cad61700 1 -- 172.21.15.26:0/3901013351 wait complete. 2024-04-21T10:30:21.361 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-21T10:30:21.361 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:30:21.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.469+0000 7f515bf9d640 1 -- 172.21.15.26:0/290765365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5154103b50 msgr2=0x7f515410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:21.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.469+0000 7f515bf9d640 1 --2- 172.21.15.26:0/290765365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5154103b50 0x7f515410a3e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f514c009ca0 tx=0x7f514c02c0f0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 -- 172.21.15.26:0/290765365 shutdown_connections 2024-04-21T10:30:21.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 --2- 172.21.15.26:0/290765365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5154103b50 0x7f515410a3e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 --2- 172.21.15.26:0/290765365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5154103190 0x7f5154103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 --2- 172.21.15.26:0/290765365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5154101fa0 0x7f51541023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 -- 172.21.15.26:0/290765365 >> 172.21.15.26:0/290765365 conn(0x7f51540fd530 msgr2=0x7f51540ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:21.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 -- 172.21.15.26:0/290765365 shutdown_connections 2024-04-21T10:30:21.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 -- 172.21.15.26:0/290765365 wait complete. 2024-04-21T10:30:21.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 Processor -- start 2024-04-21T10:30:21.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 -- start start 2024-04-21T10:30:21.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5154101fa0 0x7f5154198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:21.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5154103190 0x7f51541986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:21.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5154103b50 0x7f515419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:21.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f515410db20 con 0x7f5154103b50 2024-04-21T10:30:21.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f515410d9a0 con 0x7f5154101fa0 2024-04-21T10:30:21.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.470+0000 7f515bf9d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f515410dca0 con 0x7f5154103190 2024-04-21T10:30:21.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f515a53a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5154103b50 0x7f515419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:21.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f515a53a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5154103b50 0x7f515419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47828/0 (socket says 172.21.15.26:47828) 2024-04-21T10:30:21.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f515a53a640 1 -- 172.21.15.26:0/1177628741 learned_addr learned my addr 172.21.15.26:0/1177628741 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:21.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f5159d39640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5154101fa0 0x7f5154198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:21.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f5159538640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5154103190 0x7f51541986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:21.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f515a53a640 1 -- 172.21.15.26:0/1177628741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5154103190 msgr2=0x7f51541986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:21.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f515a53a640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5154103190 0x7f51541986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f515a53a640 1 -- 172.21.15.26:0/1177628741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5154101fa0 msgr2=0x7f5154198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:21.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f515a53a640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5154101fa0 0x7f5154198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f515a53a640 1 -- 172.21.15.26:0/1177628741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515419fd90 con 0x7f5154103b50 2024-04-21T10:30:21.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f5159d39640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5154101fa0 0x7f5154198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:21.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f515a53a640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5154103b50 0x7f515419f720 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f514c02c700 tx=0x7f514c034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:21.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f514affd640 1 -- 172.21.15.26:0/1177628741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c039040 con 0x7f5154103b50 2024-04-21T10:30:21.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f514affd640 1 -- 172.21.15.26:0/1177628741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f514c044070 con 0x7f5154103b50 2024-04-21T10:30:21.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f515bf9d640 1 -- 172.21.15.26:0/1177628741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51541a0020 con 0x7f5154103b50 2024-04-21T10:30:21.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f514affd640 1 -- 172.21.15.26:0/1177628741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c0042a0 con 0x7f5154103b50 2024-04-21T10:30:21.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.471+0000 7f515bf9d640 1 -- 172.21.15.26:0/1177628741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51541a0560 con 0x7f5154103b50 2024-04-21T10:30:21.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.472+0000 7f515bf9d640 1 -- 172.21.15.26:0/1177628741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51541092c0 con 0x7f5154103b50 2024-04-21T10:30:21.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.473+0000 7f514affd640 1 -- 172.21.15.26:0/1177628741 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f514c003970 con 0x7f5154103b50 2024-04-21T10:30:21.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.474+0000 7f514affd640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f51400a3cc0 0x7f51400a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:21.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.474+0000 7f5159d39640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f51400a3cc0 0x7f51400a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:21.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.474+0000 7f514affd640 1 -- 172.21.15.26:0/1177628741 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f514c0f3420 con 0x7f5154103b50 2024-04-21T10:30:21.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.474+0000 7f5159d39640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f51400a3cc0 0x7f51400a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f51440058c0 tx=0x7f5144005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:21.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.477+0000 7f514affd640 1 -- 172.21.15.26:0/1177628741 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f514c0bfd40 con 0x7f5154103b50 2024-04-21T10:30:21.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.666+0000 7f515bf9d640 1 -- 172.21.15.26:0/1177628741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5154062b50 con 0x7f5154103b50 2024-04-21T10:30:21.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.668+0000 7f514affd640 1 -- 172.21.15.26:0/1177628741 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22705 (secure 0 0 0) 0x7f514c0bb030 con 0x7f5154103b50 2024-04-21T10:30:21.668 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:21.669 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:30:03.297428+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:30:21.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.670+0000 7f515bf9d640 1 -- 172.21.15.26:0/1177628741 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f51400a3cc0 msgr2=0x7f51400a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:21.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.670+0000 7f515bf9d640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f51400a3cc0 0x7f51400a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f51440058c0 tx=0x7f5144005850 comp rx=0 tx=0).stop 2024-04-21T10:30:21.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.670+0000 7f515bf9d640 1 -- 172.21.15.26:0/1177628741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5154103b50 msgr2=0x7f515419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:21.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.670+0000 7f515bf9d640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5154103b50 0x7f515419f720 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f514c02c700 tx=0x7f514c034760 comp rx=0 tx=0).stop 2024-04-21T10:30:21.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.670+0000 7f515bf9d640 1 -- 172.21.15.26:0/1177628741 shutdown_connections 2024-04-21T10:30:21.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.670+0000 7f515bf9d640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f51400a3cc0 0x7f51400a6180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.670+0000 7f515bf9d640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5154103b50 0x7f515419f720 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.670+0000 7f515bf9d640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5154103190 0x7f51541986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.670+0000 7f515bf9d640 1 --2- 172.21.15.26:0/1177628741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5154101fa0 0x7f5154198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.670+0000 7f515bf9d640 1 -- 172.21.15.26:0/1177628741 >> 172.21.15.26:0/1177628741 conn(0x7f51540fd530 msgr2=0x7f5154106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:21.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.670+0000 7f515bf9d640 1 -- 172.21.15.26:0/1177628741 shutdown_connections 2024-04-21T10:30:21.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.670+0000 7f515bf9d640 1 -- 172.21.15.26:0/1177628741 wait complete. 2024-04-21T10:30:21.680 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:21 smithi026 ceph-mon[27965]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:21.680 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:21 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3901013351' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T10:30:21.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T10:24:28.176891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T10:28:59.417687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T10:29:00.234808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T10:30:21.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T10:30:21.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:21 smithi169 ceph-mon[31469]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:21.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:21 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3901013351' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T10:30:21.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:21 smithi190 ceph-mon[31423]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:21.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:21 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3901013351' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T10:30:21.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.788+0000 7f4cbeec8640 1 -- 172.21.15.26:0/810013653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cb8107c90 msgr2=0x7f4cb810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:21.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.788+0000 7f4cbeec8640 1 --2- 172.21.15.26:0/810013653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cb8107c90 0x7f4cb810a0e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009ca0 tx=0x7f4cb002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.788+0000 7f4cbeec8640 1 -- 172.21.15.26:0/810013653 shutdown_connections 2024-04-21T10:30:21.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.788+0000 7f4cbeec8640 1 --2- 172.21.15.26:0/810013653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cb8107c90 0x7f4cb810a0e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.788+0000 7f4cbeec8640 1 --2- 172.21.15.26:0/810013653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cb8105360 0x7f4cb8107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.788+0000 7f4cbeec8640 1 --2- 172.21.15.26:0/810013653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cb8102a30 0x7f4cb8104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.788+0000 7f4cbeec8640 1 -- 172.21.15.26:0/810013653 >> 172.21.15.26:0/810013653 conn(0x7f4cb80fc440 msgr2=0x7f4cb80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:21.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.788+0000 7f4cbeec8640 1 -- 172.21.15.26:0/810013653 shutdown_connections 2024-04-21T10:30:21.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.788+0000 7f4cbeec8640 1 -- 172.21.15.26:0/810013653 wait complete. 2024-04-21T10:30:21.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbeec8640 1 Processor -- start 2024-04-21T10:30:21.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbeec8640 1 -- start start 2024-04-21T10:30:21.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbeec8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cb8102a30 0x7f4cb81023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:21.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbeec8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cb8105360 0x7f4cb81009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbeec8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cb8107c90 0x7f4cb8100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbeec8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb810daf0 con 0x7f4cb8102a30 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbeec8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb810d970 con 0x7f4cb8105360 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbeec8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb810dc70 con 0x7f4cb8107c90 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbcc64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cb8102a30 0x7f4cb81023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbcc64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cb8102a30 0x7f4cb81023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47850/0 (socket says 172.21.15.26:47850) 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbcc64640 1 -- 172.21.15.26:0/3750000802 learned_addr learned my addr 172.21.15.26:0/3750000802 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cb7fff640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cb8105360 0x7f4cb81009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbcc64640 1 -- 172.21.15.26:0/3750000802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cb8107c90 msgr2=0x7f4cb8100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbcc64640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cb8107c90 0x7f4cb8100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbcc64640 1 -- 172.21.15.26:0/3750000802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cb8105360 msgr2=0x7f4cb81009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbcc64640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cb8105360 0x7f4cb81009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cbcc64640 1 -- 172.21.15.26:0/3750000802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb81017b0 con 0x7f4cb8102a30 2024-04-21T10:30:21.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.789+0000 7f4cb7fff640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cb8105360 0x7f4cb81009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:21.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.790+0000 7f4cbcc64640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cb8102a30 0x7f4cb81023a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00d950 tx=0x7f4cac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:21.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.790+0000 7f4cb5ffb640 1 -- 172.21.15.26:0/3750000802 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac017070 con 0x7f4cb8102a30 2024-04-21T10:30:21.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.790+0000 7f4cb5ffb640 1 -- 172.21.15.26:0/3750000802 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cac009c30 con 0x7f4cb8102a30 2024-04-21T10:30:21.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.790+0000 7f4cbeec8640 1 -- 172.21.15.26:0/3750000802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb81a22e0 con 0x7f4cb8102a30 2024-04-21T10:30:21.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.790+0000 7f4cb5ffb640 1 -- 172.21.15.26:0/3750000802 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac00f620 con 0x7f4cb8102a30 2024-04-21T10:30:21.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.790+0000 7f4cbeec8640 1 -- 172.21.15.26:0/3750000802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb81a2830 con 0x7f4cb8102a30 2024-04-21T10:30:21.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.791+0000 7f4cbeec8640 1 -- 172.21.15.26:0/3750000802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb80fdf30 con 0x7f4cb8102a30 2024-04-21T10:30:21.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.792+0000 7f4cb5ffb640 1 -- 172.21.15.26:0/3750000802 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4cac010460 con 0x7f4cb8102a30 2024-04-21T10:30:21.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.793+0000 7f4cb5ffb640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4ca80a3c40 0x7f4ca80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:21.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.793+0000 7f4cb7fff640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4ca80a3c40 0x7f4ca80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:21.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.793+0000 7f4cb5ffb640 1 -- 172.21.15.26:0/3750000802 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f4cac0d02e0 con 0x7f4cb8102a30 2024-04-21T10:30:21.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.793+0000 7f4cb7fff640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4ca80a3c40 0x7f4ca80a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40058c0 tx=0x7f4ca4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:21.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.798+0000 7f4cb5ffb640 1 -- 172.21.15.26:0/3750000802 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cac09caf0 con 0x7f4cb8102a30 2024-04-21T10:30:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.986+0000 7f4cbeec8640 1 -- 172.21.15.26:0/3750000802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4cb8062b50 con 0x7f4cb8102a30 2024-04-21T10:30:21.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.987+0000 7f4cb5ffb640 1 -- 172.21.15.26:0/3750000802 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f4cac009de0 con 0x7f4cb8102a30 2024-04-21T10:30:21.986 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-21T10:30:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.989+0000 7f4cbeec8640 1 -- 172.21.15.26:0/3750000802 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4ca80a3c40 msgr2=0x7f4ca80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.989+0000 7f4cbeec8640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4ca80a3c40 0x7f4ca80a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40058c0 tx=0x7f4ca4005850 comp rx=0 tx=0).stop 2024-04-21T10:30:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.989+0000 7f4cbeec8640 1 -- 172.21.15.26:0/3750000802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cb8102a30 msgr2=0x7f4cb81023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.989+0000 7f4cbeec8640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cb8102a30 0x7f4cb81023a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00d950 tx=0x7f4cac00de10 comp rx=0 tx=0).stop 2024-04-21T10:30:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.989+0000 7f4cbeec8640 1 -- 172.21.15.26:0/3750000802 shutdown_connections 2024-04-21T10:30:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.989+0000 7f4cbeec8640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4ca80a3c40 0x7f4ca80a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.989+0000 7f4cbeec8640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cb8107c90 0x7f4cb8100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.989+0000 7f4cbeec8640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cb8105360 0x7f4cb81009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.989+0000 7f4cbeec8640 1 --2- 172.21.15.26:0/3750000802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cb8102a30 0x7f4cb81023a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.989+0000 7f4cbeec8640 1 -- 172.21.15.26:0/3750000802 >> 172.21.15.26:0/3750000802 conn(0x7f4cb80fc440 msgr2=0x7f4cb8103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.989+0000 7f4cbeec8640 1 -- 172.21.15.26:0/3750000802 shutdown_connections 2024-04-21T10:30:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:21.989+0000 7f4cbeec8640 1 -- 172.21.15.26:0/3750000802 wait complete. 2024-04-21T10:30:22.000 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T10:30:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.106+0000 7f80df843640 1 -- 172.21.15.26:0/2301332389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d8100e60 msgr2=0x7f80d8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.106+0000 7f80df843640 1 --2- 172.21.15.26:0/2301332389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d8100e60 0x7f80d8101280 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009900 tx=0x7f80d402bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 -- 172.21.15.26:0/2301332389 shutdown_connections 2024-04-21T10:30:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 --2- 172.21.15.26:0/2301332389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d8102a00 0x7f80d8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 --2- 172.21.15.26:0/2301332389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80d8102040 0x7f80d81024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 --2- 172.21.15.26:0/2301332389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d8100e60 0x7f80d8101280 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 -- 172.21.15.26:0/2301332389 >> 172.21.15.26:0/2301332389 conn(0x7f80d80fc420 msgr2=0x7f80d80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 -- 172.21.15.26:0/2301332389 shutdown_connections 2024-04-21T10:30:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 -- 172.21.15.26:0/2301332389 wait complete. 2024-04-21T10:30:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 Processor -- start 2024-04-21T10:30:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 -- start start 2024-04-21T10:30:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d8100e60 0x7f80d8197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80d8102040 0x7f80d8197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d8102a00 0x7f80d819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f80d810c9d0 con 0x7f80d8102a00 2024-04-21T10:30:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f80d810c850 con 0x7f80d8102040 2024-04-21T10:30:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80df843640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f80d810cb50 con 0x7f80d8100e60 2024-04-21T10:30:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80ddde0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d8102a00 0x7f80d819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.107+0000 7f80ddde0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d8102a00 0x7f80d819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47878/0 (socket says 172.21.15.26:47878) 2024-04-21T10:30:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80ddde0640 1 -- 172.21.15.26:0/2994834 learned_addr learned my addr 172.21.15.26:0/2994834 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:22.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80dd5df640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d8100e60 0x7f80d8197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:22.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80dcdde640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80d8102040 0x7f80d8197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:22.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80ddde0640 1 -- 172.21.15.26:0/2994834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d8100e60 msgr2=0x7f80d8197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80ddde0640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d8100e60 0x7f80d8197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80ddde0640 1 -- 172.21.15.26:0/2994834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80d8102040 msgr2=0x7f80d8197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80ddde0640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80d8102040 0x7f80d8197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80ddde0640 1 -- 172.21.15.26:0/2994834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d819ed30 con 0x7f80d8102a00 2024-04-21T10:30:22.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80ddde0640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d8102a00 0x7f80d819e6c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f80d000eaa0 tx=0x7f80d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:22.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80ce7fc640 1 -- 172.21.15.26:0/2994834 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d000cdb0 con 0x7f80d8102a00 2024-04-21T10:30:22.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80ce7fc640 1 -- 172.21.15.26:0/2994834 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80d0005020 con 0x7f80d8102a00 2024-04-21T10:30:22.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80df843640 1 -- 172.21.15.26:0/2994834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d819efc0 con 0x7f80d8102a00 2024-04-21T10:30:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80ce7fc640 1 -- 172.21.15.26:0/2994834 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d00116d0 con 0x7f80d8102a00 2024-04-21T10:30:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.108+0000 7f80df843640 1 -- 172.21.15.26:0/2994834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d819f490 con 0x7f80d8102a00 2024-04-21T10:30:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.109+0000 7f80df843640 1 -- 172.21.15.26:0/2994834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d8108320 con 0x7f80d8102a00 2024-04-21T10:30:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.110+0000 7f80ce7fc640 1 -- 172.21.15.26:0/2994834 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f80d00051d0 con 0x7f80d8102a00 2024-04-21T10:30:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.111+0000 7f80ce7fc640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f80c40a3c40 0x7f80c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.111+0000 7f80dd5df640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f80c40a3c40 0x7f80c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.111+0000 7f80ce7fc640 1 -- 172.21.15.26:0/2994834 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f80d00d06f0 con 0x7f80d8102a00 2024-04-21T10:30:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.111+0000 7f80dd5df640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f80c40a3c40 0x7f80c40a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80d4005ab0 tx=0x7f80d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:22.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.115+0000 7f80ce7fc640 1 -- 172.21.15.26:0/2994834 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80d0016030 con 0x7f80d8102a00 2024-04-21T10:30:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.302+0000 7f80df843640 1 -- 172.21.15.26:0/2994834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f80d8062b50 con 0x7f80d8102a00 2024-04-21T10:30:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.302+0000 7f80ce7fc640 1 -- 172.21.15.26:0/2994834 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f80d009c910 con 0x7f80d8102a00 2024-04-21T10:30:22.302 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-21T10:30:22.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.304+0000 7f80df843640 1 -- 172.21.15.26:0/2994834 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f80c40a3c40 msgr2=0x7f80c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.304+0000 7f80df843640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f80c40a3c40 0x7f80c40a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80d4005ab0 tx=0x7f80d4005fd0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.304+0000 7f80df843640 1 -- 172.21.15.26:0/2994834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d8102a00 msgr2=0x7f80d819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.304+0000 7f80df843640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d8102a00 0x7f80d819e6c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f80d000eaa0 tx=0x7f80d000ef60 comp rx=0 tx=0).stop 2024-04-21T10:30:22.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.304+0000 7f80df843640 1 -- 172.21.15.26:0/2994834 shutdown_connections 2024-04-21T10:30:22.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.304+0000 7f80df843640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f80c40a3c40 0x7f80c40a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.304+0000 7f80df843640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d8102a00 0x7f80d819e6c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.304+0000 7f80df843640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80d8102040 0x7f80d8197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.304+0000 7f80df843640 1 --2- 172.21.15.26:0/2994834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80d8100e60 0x7f80d8197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.304+0000 7f80df843640 1 -- 172.21.15.26:0/2994834 >> 172.21.15.26:0/2994834 conn(0x7f80d80fc420 msgr2=0x7f80d81054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:22.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.304+0000 7f80df843640 1 -- 172.21.15.26:0/2994834 shutdown_connections 2024-04-21T10:30:22.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.304+0000 7f80df843640 1 -- 172.21.15.26:0/2994834 wait complete. 2024-04-21T10:30:22.315 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T10:30:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.421+0000 7f7b4983a640 1 -- 172.21.15.26:0/94320018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b441032f0 msgr2=0x7f7b44103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.421+0000 7f7b4983a640 1 --2- 172.21.15.26:0/94320018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b441032f0 0x7f7b44103770 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009930 tx=0x7f7b3802bc20 comp rx=0 tx=0).stop 2024-04-21T10:30:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.421+0000 7f7b4983a640 1 -- 172.21.15.26:0/94320018 shutdown_connections 2024-04-21T10:30:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.421+0000 7f7b4983a640 1 --2- 172.21.15.26:0/94320018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b44103cb0 0x7f7b4410a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.421+0000 7f7b4983a640 1 --2- 172.21.15.26:0/94320018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b441032f0 0x7f7b44103770 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.421+0000 7f7b4983a640 1 --2- 172.21.15.26:0/94320018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b44100280 0x7f7b441006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.421+0000 7f7b4983a640 1 -- 172.21.15.26:0/94320018 >> 172.21.15.26:0/94320018 conn(0x7f7b440fbb90 msgr2=0x7f7b440fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.421+0000 7f7b4983a640 1 -- 172.21.15.26:0/94320018 shutdown_connections 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.421+0000 7f7b4983a640 1 -- 172.21.15.26:0/94320018 wait complete. 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.421+0000 7f7b4983a640 1 Processor -- start 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b4983a640 1 -- start start 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b4983a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b44100280 0x7f7b441981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b4983a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b441032f0 0x7f7b44198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b4983a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44103cb0 0x7f7b4419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b4983a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4410db80 con 0x7f7b44103cb0 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b4983a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4410da00 con 0x7f7b441032f0 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b4983a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4410dd00 con 0x7f7b44100280 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44103cb0 0x7f7b4419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44103cb0 0x7f7b4419f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47894/0 (socket says 172.21.15.26:47894) 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b437fe640 1 -- 172.21.15.26:0/2172945994 learned_addr learned my addr 172.21.15.26:0/2172945994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b42ffd640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b44100280 0x7f7b441981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b437fe640 1 -- 172.21.15.26:0/2172945994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b44100280 msgr2=0x7f7b441981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b437fe640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b44100280 0x7f7b441981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b437fe640 1 -- 172.21.15.26:0/2172945994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b441032f0 msgr2=0x7f7b44198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b427fc640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b441032f0 0x7f7b44198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b437fe640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b441032f0 0x7f7b44198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b437fe640 1 -- 172.21.15.26:0/2172945994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4419fe10 con 0x7f7b44103cb0 2024-04-21T10:30:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b42ffd640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b44100280 0x7f7b441981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b437fe640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44103cb0 0x7f7b4419f7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00b970 tx=0x7f7b3c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.422+0000 7f7b48838640 1 -- 172.21.15.26:0/2172945994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c019070 con 0x7f7b44103cb0 2024-04-21T10:30:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.423+0000 7f7b48838640 1 -- 172.21.15.26:0/2172945994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b3c014440 con 0x7f7b44103cb0 2024-04-21T10:30:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.423+0000 7f7b4983a640 1 -- 172.21.15.26:0/2172945994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b441a0100 con 0x7f7b44103cb0 2024-04-21T10:30:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.423+0000 7f7b48838640 1 -- 172.21.15.26:0/2172945994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0075d0 con 0x7f7b44103cb0 2024-04-21T10:30:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.423+0000 7f7b4983a640 1 -- 172.21.15.26:0/2172945994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b441a06d0 con 0x7f7b44103cb0 2024-04-21T10:30:22.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.424+0000 7f7b4983a640 1 -- 172.21.15.26:0/2172945994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b44109420 con 0x7f7b44103cb0 2024-04-21T10:30:22.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.425+0000 7f7b48838640 1 -- 172.21.15.26:0/2172945994 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7b3c01c070 con 0x7f7b44103cb0 2024-04-21T10:30:22.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.426+0000 7f7b48838640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7b2c0a3cc0 0x7f7b2c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:22.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.426+0000 7f7b42ffd640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7b2c0a3cc0 0x7f7b2c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:22.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.426+0000 7f7b48838640 1 -- 172.21.15.26:0/2172945994 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f7b3c0cfa60 con 0x7f7b44103cb0 2024-04-21T10:30:22.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.426+0000 7f7b42ffd640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7b2c0a3cc0 0x7f7b2c0a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7b4406a2d0 tx=0x7f7b34005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:22.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.430+0000 7f7b48838640 1 -- 172.21.15.26:0/2172945994 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b3c09c3e0 con 0x7f7b44103cb0 2024-04-21T10:30:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.616+0000 7f7b4983a640 1 -- 172.21.15.26:0/2172945994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7b44062b50 con 0x7f7b44103cb0 2024-04-21T10:30:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.616+0000 7f7b48838640 1 -- 172.21.15.26:0/2172945994 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f7b3c09bce0 con 0x7f7b44103cb0 2024-04-21T10:30:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-21T10:30:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.618+0000 7f7b4983a640 1 -- 172.21.15.26:0/2172945994 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7b2c0a3cc0 msgr2=0x7f7b2c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.618+0000 7f7b4983a640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7b2c0a3cc0 0x7f7b2c0a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7b4406a2d0 tx=0x7f7b34005db0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.618+0000 7f7b4983a640 1 -- 172.21.15.26:0/2172945994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44103cb0 msgr2=0x7f7b4419f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.618+0000 7f7b4983a640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44103cb0 0x7f7b4419f7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00b970 tx=0x7f7b3c00be30 comp rx=0 tx=0).stop 2024-04-21T10:30:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.618+0000 7f7b4983a640 1 -- 172.21.15.26:0/2172945994 shutdown_connections 2024-04-21T10:30:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.618+0000 7f7b4983a640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7b2c0a3cc0 0x7f7b2c0a6180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.618+0000 7f7b4983a640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44103cb0 0x7f7b4419f7a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.618+0000 7f7b4983a640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b441032f0 0x7f7b44198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.618+0000 7f7b4983a640 1 --2- 172.21.15.26:0/2172945994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b44100280 0x7f7b441981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.618+0000 7f7b4983a640 1 -- 172.21.15.26:0/2172945994 >> 172.21.15.26:0/2172945994 conn(0x7f7b440fbb90 msgr2=0x7f7b441065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.618+0000 7f7b4983a640 1 -- 172.21.15.26:0/2172945994 shutdown_connections 2024-04-21T10:30:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.618+0000 7f7b4983a640 1 -- 172.21.15.26:0/2172945994 wait complete. 2024-04-21T10:30:22.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:22 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1177628741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:22.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:22 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3750000802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:30:22.628 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:22 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2994834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:30:22.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:30:22.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:22 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1177628741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:22.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:22 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3750000802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:30:22.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:22 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2994834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:30:22.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.731+0000 7f4e45dce640 1 -- 172.21.15.26:0/4151859613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40103190 msgr2=0x7f4e40103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.731+0000 7f4e45dce640 1 --2- 172.21.15.26:0/4151859613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40103190 0x7f4e40103610 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009900 tx=0x7f4e3402bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.731+0000 7f4e45dce640 1 -- 172.21.15.26:0/4151859613 shutdown_connections 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.731+0000 7f4e45dce640 1 --2- 172.21.15.26:0/4151859613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e40103b50 0x7f4e4010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.731+0000 7f4e45dce640 1 --2- 172.21.15.26:0/4151859613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40103190 0x7f4e40103610 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.731+0000 7f4e45dce640 1 --2- 172.21.15.26:0/4151859613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e40101fa0 0x7f4e401023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.731+0000 7f4e45dce640 1 -- 172.21.15.26:0/4151859613 >> 172.21.15.26:0/4151859613 conn(0x7f4e400fd530 msgr2=0x7f4e400ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.732+0000 7f4e45dce640 1 -- 172.21.15.26:0/4151859613 shutdown_connections 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.732+0000 7f4e45dce640 1 -- 172.21.15.26:0/4151859613 wait complete. 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.732+0000 7f4e45dce640 1 Processor -- start 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.732+0000 7f4e45dce640 1 -- start start 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.732+0000 7f4e45dce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e40101fa0 0x7f4e401981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.732+0000 7f4e45dce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40103190 0x7f4e401986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.732+0000 7f4e45dce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e40103b50 0x7f4e4019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.732+0000 7f4e45dce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4010dae0 con 0x7f4e40103190 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.732+0000 7f4e45dce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4010d960 con 0x7f4e40101fa0 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.732+0000 7f4e45dce640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4010dc60 con 0x7f4e40103b50 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.732+0000 7f4e3effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40103190 0x7f4e401986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40103190 0x7f4e401986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47906/0 (socket says 172.21.15.26:47906) 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3effd640 1 -- 172.21.15.26:0/2840251133 learned_addr learned my addr 172.21.15.26:0/2840251133 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3f7fe640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e40101fa0 0x7f4e401981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3ffff640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e40103b50 0x7f4e4019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3effd640 1 -- 172.21.15.26:0/2840251133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e40103b50 msgr2=0x7f4e4019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3effd640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e40103b50 0x7f4e4019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3effd640 1 -- 172.21.15.26:0/2840251133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e40101fa0 msgr2=0x7f4e401981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3effd640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e40101fa0 0x7f4e401981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3effd640 1 -- 172.21.15.26:0/2840251133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4019fdc0 con 0x7f4e40103190 2024-04-21T10:30:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3ffff640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e40103b50 0x7f4e4019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3f7fe640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e40101fa0 0x7f4e401981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3effd640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40103190 0x7f4e401986e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4e34006010 tx=0x7f4e340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3cff9640 1 -- 172.21.15.26:0/2840251133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34043020 con 0x7f4e40103190 2024-04-21T10:30:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e45dce640 1 -- 172.21.15.26:0/2840251133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e401a0050 con 0x7f4e40103190 2024-04-21T10:30:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3cff9640 1 -- 172.21.15.26:0/2840251133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e34004aa0 con 0x7f4e40103190 2024-04-21T10:30:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e3cff9640 1 -- 172.21.15.26:0/2840251133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34003c70 con 0x7f4e40103190 2024-04-21T10:30:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.733+0000 7f4e45dce640 1 -- 172.21.15.26:0/2840251133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e401a05f0 con 0x7f4e40103190 2024-04-21T10:30:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.734+0000 7f4e45dce640 1 -- 172.21.15.26:0/2840251133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e401092e0 con 0x7f4e40103190 2024-04-21T10:30:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.736+0000 7f4e3cff9640 1 -- 172.21.15.26:0/2840251133 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4e34046020 con 0x7f4e40103190 2024-04-21T10:30:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.736+0000 7f4e3cff9640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4e280a3c40 0x7f4e280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.737+0000 7f4e3f7fe640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4e280a3c40 0x7f4e280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.737+0000 7f4e3cff9640 1 -- 172.21.15.26:0/2840251133 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f4e340f2fa0 con 0x7f4e40103190 2024-04-21T10:30:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.737+0000 7f4e3f7fe640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4e280a3c40 0x7f4e280a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4e300058c0 tx=0x7f4e30005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.740+0000 7f4e3cff9640 1 -- 172.21.15.26:0/2840251133 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e3404d020 con 0x7f4e40103190 2024-04-21T10:30:22.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:22 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1177628741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:22.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:22 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3750000802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:30:22.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:22 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2994834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:30:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.965+0000 7f4e45dce640 1 -- 172.21.15.26:0/2840251133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e4002d5c0 con 0x7f4e40103190 2024-04-21T10:30:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.966+0000 7f4e3cff9640 1 -- 172.21.15.26:0/2840251133 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11884 (secure 0 0 0) 0x7f4e340bf920 con 0x7f4e40103190 2024-04-21T10:30:22.966 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:22.966 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:29:05.806617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24478,24488,24496]},"id":1}]} 2024-04-21T10:30:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 20 2024-04-21T10:30:22.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.968+0000 7f4e45dce640 1 -- 172.21.15.26:0/2840251133 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4e280a3c40 msgr2=0x7f4e280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.968+0000 7f4e45dce640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4e280a3c40 0x7f4e280a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4e300058c0 tx=0x7f4e30005850 comp rx=0 tx=0).stop 2024-04-21T10:30:22.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.968+0000 7f4e45dce640 1 -- 172.21.15.26:0/2840251133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40103190 msgr2=0x7f4e401986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:22.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.968+0000 7f4e45dce640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40103190 0x7f4e401986e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4e34006010 tx=0x7f4e340042f0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.968+0000 7f4e45dce640 1 -- 172.21.15.26:0/2840251133 shutdown_connections 2024-04-21T10:30:22.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.968+0000 7f4e45dce640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f4e280a3c40 0x7f4e280a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.968+0000 7f4e45dce640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e40103b50 0x7f4e4019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.968+0000 7f4e45dce640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40103190 0x7f4e401986e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.968+0000 7f4e45dce640 1 --2- 172.21.15.26:0/2840251133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e40101fa0 0x7f4e401981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:22.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.968+0000 7f4e45dce640 1 -- 172.21.15.26:0/2840251133 >> 172.21.15.26:0/2840251133 conn(0x7f4e400fd530 msgr2=0x7f4e401064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:22.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.968+0000 7f4e45dce640 1 -- 172.21.15.26:0/2840251133 shutdown_connections 2024-04-21T10:30:22.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:22.968+0000 7f4e45dce640 1 -- 172.21.15.26:0/2840251133 wait complete. 2024-04-21T10:30:22.980 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:30:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.081+0000 7fc1021f4640 1 -- 172.21.15.26:0/1102389803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc102a50 msgr2=0x7fc0fc104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.081+0000 7fc1021f4640 1 --2- 172.21.15.26:0/1102389803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc102a50 0x7fc0fc104e40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009900 tx=0x7fc0ec02bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.081+0000 7fc1021f4640 1 -- 172.21.15.26:0/1102389803 shutdown_connections 2024-04-21T10:30:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.081+0000 7fc1021f4640 1 --2- 172.21.15.26:0/1102389803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0fc107cb0 0x7fc0fc10a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.081+0000 7fc1021f4640 1 --2- 172.21.15.26:0/1102389803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0fc105380 0x7fc0fc107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.081+0000 7fc1021f4640 1 --2- 172.21.15.26:0/1102389803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc102a50 0x7fc0fc104e40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.081+0000 7fc1021f4640 1 -- 172.21.15.26:0/1102389803 >> 172.21.15.26:0/1102389803 conn(0x7fc0fc0fc3e0 msgr2=0x7fc0fc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.081+0000 7fc1021f4640 1 -- 172.21.15.26:0/1102389803 shutdown_connections 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.081+0000 7fc1021f4640 1 -- 172.21.15.26:0/1102389803 wait complete. 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.082+0000 7fc1021f4640 1 Processor -- start 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.082+0000 7fc1021f4640 1 -- start start 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.082+0000 7fc1021f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0fc102a50 0x7fc0fc1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.082+0000 7fc1021f4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0fc105380 0x7fc0fc198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.082+0000 7fc1021f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc107cb0 0x7fc0fc19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.082+0000 7fc1021f4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc10dab0 con 0x7fc0fc107cb0 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.082+0000 7fc1021f4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc10d930 con 0x7fc0fc105380 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.082+0000 7fc1021f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc10dc30 con 0x7fc0fc102a50 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.082+0000 7fc0fbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc107cb0 0x7fc0fc19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.082+0000 7fc0fbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc107cb0 0x7fc0fc19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47920/0 (socket says 172.21.15.26:47920) 2024-04-21T10:30:23.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.082+0000 7fc0fbfff640 1 -- 172.21.15.26:0/938265962 learned_addr learned my addr 172.21.15.26:0/938265962 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc0fb7fe640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0fc102a50 0x7fc0fc1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc0faffd640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0fc105380 0x7fc0fc198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc0fbfff640 1 -- 172.21.15.26:0/938265962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0fc102a50 msgr2=0x7fc0fc1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc0fbfff640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0fc102a50 0x7fc0fc1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc0fbfff640 1 -- 172.21.15.26:0/938265962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0fc105380 msgr2=0x7fc0fc198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc0fbfff640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0fc105380 0x7fc0fc198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc0fbfff640 1 -- 172.21.15.26:0/938265962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc19fe00 con 0x7fc0fc107cb0 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc0faffd640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0fc105380 0x7fc0fc198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc0fbfff640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc107cb0 0x7fc0fc19f790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400eb10 tx=0x7fc0f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc0f8ff9640 1 -- 172.21.15.26:0/938265962 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4004b20 con 0x7fc0fc107cb0 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc1021f4640 1 -- 172.21.15.26:0/938265962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0fc1a00f0 con 0x7fc0fc107cb0 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc0f8ff9640 1 -- 172.21.15.26:0/938265962 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f4004cc0 con 0x7fc0fc107cb0 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc0f8ff9640 1 -- 172.21.15.26:0/938265962 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4010670 con 0x7fc0fc107cb0 2024-04-21T10:30:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.083+0000 7fc1021f4640 1 -- 172.21.15.26:0/938265962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0fc1a0640 con 0x7fc0fc107cb0 2024-04-21T10:30:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.084+0000 7fc1021f4640 1 -- 172.21.15.26:0/938265962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0fc0fded0 con 0x7fc0fc107cb0 2024-04-21T10:30:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.086+0000 7fc0f8ff9640 1 -- 172.21.15.26:0/938265962 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc0f4005000 con 0x7fc0fc107cb0 2024-04-21T10:30:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.086+0000 7fc0f8ff9640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc0e40a3c40 0x7fc0e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.087+0000 7fc0fb7fe640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc0e40a3c40 0x7fc0e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.087+0000 7fc0f8ff9640 1 -- 172.21.15.26:0/938265962 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fc0f40d0e10 con 0x7fc0fc107cb0 2024-04-21T10:30:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.087+0000 7fc0fb7fe640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc0e40a3c40 0x7fc0e40a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009230 tx=0x7fc0ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:23.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.090+0000 7fc0f8ff9640 1 -- 172.21.15.26:0/938265962 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0f4053ff0 con 0x7fc0fc107cb0 2024-04-21T10:30:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.278+0000 7fc1021f4640 1 -- 172.21.15.26:0/938265962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0fc062b50 con 0x7fc0fc107cb0 2024-04-21T10:30:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.280+0000 7fc0f8ff9640 1 -- 172.21.15.26:0/938265962 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22705 (secure 0 0 0) 0x7fc0f409d680 con 0x7fc0fc107cb0 2024-04-21T10:30:23.280 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:23.280 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:30:03.297428+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:30:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.282+0000 7fc1021f4640 1 -- 172.21.15.26:0/938265962 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc0e40a3c40 msgr2=0x7fc0e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.282+0000 7fc1021f4640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc0e40a3c40 0x7fc0e40a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009230 tx=0x7fc0ec005fd0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.282+0000 7fc1021f4640 1 -- 172.21.15.26:0/938265962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc107cb0 msgr2=0x7fc0fc19f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.282+0000 7fc1021f4640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc107cb0 0x7fc0fc19f790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400eb10 tx=0x7fc0f400c2d0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.282+0000 7fc1021f4640 1 -- 172.21.15.26:0/938265962 shutdown_connections 2024-04-21T10:30:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.282+0000 7fc1021f4640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fc0e40a3c40 0x7fc0e40a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.282+0000 7fc1021f4640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0fc107cb0 0x7fc0fc19f790 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.282+0000 7fc1021f4640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0fc105380 0x7fc0fc198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.282+0000 7fc1021f4640 1 --2- 172.21.15.26:0/938265962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0fc102a50 0x7fc0fc1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.282+0000 7fc1021f4640 1 -- 172.21.15.26:0/938265962 >> 172.21.15.26:0/938265962 conn(0x7fc0fc0fc3e0 msgr2=0x7fc0fc1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.282+0000 7fc1021f4640 1 -- 172.21.15.26:0/938265962 shutdown_connections 2024-04-21T10:30:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.282+0000 7fc1021f4640 1 -- 172.21.15.26:0/938265962 wait complete. 2024-04-21T10:30:23.294 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T10:24:28.176891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T10:28:59.417687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T10:29:00.234808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T10:30:23.294 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T10:30:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.399+0000 7f0815c25640 1 -- 172.21.15.26:0/489752963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08101008a0 msgr2=0x7f0810100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.399+0000 7f0815c25640 1 --2- 172.21.15.26:0/489752963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08101008a0 0x7f0810100cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0800009900 tx=0x7f080002bcd0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.399+0000 7f0815c25640 1 -- 172.21.15.26:0/489752963 shutdown_connections 2024-04-21T10:30:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.399+0000 7f0815c25640 1 --2- 172.21.15.26:0/489752963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0810101ba0 0x7f0810102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.399+0000 7f0815c25640 1 --2- 172.21.15.26:0/489752963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0810101200 0x7f0810101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.399+0000 7f0815c25640 1 --2- 172.21.15.26:0/489752963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08101008a0 0x7f0810100cc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.399+0000 7f0815c25640 1 -- 172.21.15.26:0/489752963 >> 172.21.15.26:0/489752963 conn(0x7f08100fc440 msgr2=0x7f08100fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.399+0000 7f0815c25640 1 -- 172.21.15.26:0/489752963 shutdown_connections 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.399+0000 7f0815c25640 1 -- 172.21.15.26:0/489752963 wait complete. 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.400+0000 7f0815c25640 1 Processor -- start 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.400+0000 7f0815c25640 1 -- start start 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.400+0000 7f0815c25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08101008a0 0x7f081019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.400+0000 7f0815c25640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0810101200 0x7f081019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.400+0000 7f0815c25640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0810101ba0 0x7f08101a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.400+0000 7f0815c25640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f081010fd10 con 0x7f08101008a0 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.400+0000 7f0815c25640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f081010fb90 con 0x7f0810101ba0 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.400+0000 7f0815c25640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f081010fe90 con 0x7f0810101200 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.400+0000 7f080f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08101008a0 0x7f081019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:23.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.400+0000 7f080f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08101008a0 0x7f081019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47936/0 (socket says 172.21.15.26:47936) 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.400+0000 7f080f7fe640 1 -- 172.21.15.26:0/3394566064 learned_addr learned my addr 172.21.15.26:0/3394566064 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.400+0000 7f080effd640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0810101200 0x7f081019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.401+0000 7f080f7fe640 1 -- 172.21.15.26:0/3394566064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0810101200 msgr2=0x7f081019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.401+0000 7f080f7fe640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0810101200 0x7f081019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.401+0000 7f080ffff640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0810101ba0 0x7f08101a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.401+0000 7f080f7fe640 1 -- 172.21.15.26:0/3394566064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0810101ba0 msgr2=0x7f08101a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.401+0000 7f080f7fe640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0810101ba0 0x7f08101a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.401+0000 7f080f7fe640 1 -- 172.21.15.26:0/3394566064 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08101a1fd0 con 0x7f08101008a0 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.401+0000 7f080f7fe640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08101008a0 0x7f081019a3b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0800000c00 tx=0x7f080002dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.401+0000 7f080cff9640 1 -- 172.21.15.26:0/3394566064 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800043020 con 0x7f08101008a0 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.401+0000 7f0815c25640 1 -- 172.21.15.26:0/3394566064 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08101a2260 con 0x7f08101008a0 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.401+0000 7f080cff9640 1 -- 172.21.15.26:0/3394566064 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f080002de30 con 0x7f08101008a0 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.401+0000 7f080cff9640 1 -- 172.21.15.26:0/3394566064 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800003b90 con 0x7f08101008a0 2024-04-21T10:30:23.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.401+0000 7f0815c25640 1 -- 172.21.15.26:0/3394566064 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08101a2800 con 0x7f08101008a0 2024-04-21T10:30:23.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.402+0000 7f0815c25640 1 -- 172.21.15.26:0/3394566064 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08100691c0 con 0x7f08101008a0 2024-04-21T10:30:23.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.403+0000 7f080cff9640 1 -- 172.21.15.26:0/3394566064 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0800046020 con 0x7f08101008a0 2024-04-21T10:30:23.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.404+0000 7f080cff9640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f07f80a3c40 0x7f07f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:23.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.404+0000 7f080effd640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f07f80a3c40 0x7f07f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:23.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.404+0000 7f080cff9640 1 -- 172.21.15.26:0/3394566064 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f08000f27e0 con 0x7f08101008a0 2024-04-21T10:30:23.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.404+0000 7f080effd640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f07f80a3c40 0x7f07f80a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f081019b8c0 tx=0x7f080400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:23.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.408+0000 7f080cff9640 1 -- 172.21.15.26:0/3394566064 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f080004d020 con 0x7f08101008a0 2024-04-21T10:30:23.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.594+0000 7f0815c25640 1 -- 172.21.15.26:0/3394566064 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f081010b5f0 con 0x7f08101008a0 2024-04-21T10:30:23.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.594+0000 7f080cff9640 1 -- 172.21.15.26:0/3394566064 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f081010b5f0 con 0x7f08101008a0 2024-04-21T10:30:23.594 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-21T10:30:23.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.596+0000 7f0815c25640 1 -- 172.21.15.26:0/3394566064 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f07f80a3c40 msgr2=0x7f07f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.596+0000 7f0815c25640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f07f80a3c40 0x7f07f80a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f081019b8c0 tx=0x7f080400a2d0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.596+0000 7f0815c25640 1 -- 172.21.15.26:0/3394566064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08101008a0 msgr2=0x7f081019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.596+0000 7f0815c25640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08101008a0 0x7f081019a3b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0800000c00 tx=0x7f080002dc40 comp rx=0 tx=0).stop 2024-04-21T10:30:23.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.596+0000 7f0815c25640 1 -- 172.21.15.26:0/3394566064 shutdown_connections 2024-04-21T10:30:23.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.596+0000 7f0815c25640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f07f80a3c40 0x7f07f80a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.596+0000 7f0815c25640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0810101ba0 0x7f08101a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.596+0000 7f0815c25640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0810101200 0x7f081019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.596+0000 7f0815c25640 1 --2- 172.21.15.26:0/3394566064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08101008a0 0x7f081019a3b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.596+0000 7f0815c25640 1 -- 172.21.15.26:0/3394566064 >> 172.21.15.26:0/3394566064 conn(0x7f08100fc440 msgr2=0x7f08100fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:23.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.597+0000 7f0815c25640 1 -- 172.21.15.26:0/3394566064 shutdown_connections 2024-04-21T10:30:23.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.597+0000 7f0815c25640 1 -- 172.21.15.26:0/3394566064 wait complete. 2024-04-21T10:30:23.606 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:23 smithi026 ceph-mon[27965]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:30:23.607 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:23 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2172945994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:30:23.607 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:23 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2840251133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:23.607 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:23 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/938265962' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:23.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T10:30:23.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:23 smithi169 ceph-mon[31469]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:30:23.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:23 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2172945994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:30:23.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:23 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2840251133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:23.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:23 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/938265962' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.713+0000 7f5849d98640 1 -- 172.21.15.26:0/1986086939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5844102ad0 msgr2=0x7f5844104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.713+0000 7f5849d98640 1 --2- 172.21.15.26:0/1986086939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5844102ad0 0x7f5844104ec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5834009900 tx=0x7f583402bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.713+0000 7f5849d98640 1 -- 172.21.15.26:0/1986086939 shutdown_connections 2024-04-21T10:30:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.713+0000 7f5849d98640 1 --2- 172.21.15.26:0/1986086939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5844107d30 0x7f584410a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.713+0000 7f5849d98640 1 --2- 172.21.15.26:0/1986086939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5844105400 0x7f58441077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.713+0000 7f5849d98640 1 --2- 172.21.15.26:0/1986086939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5844102ad0 0x7f5844104ec0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.713+0000 7f5849d98640 1 -- 172.21.15.26:0/1986086939 >> 172.21.15.26:0/1986086939 conn(0x7f58440fc440 msgr2=0x7f58440fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.713+0000 7f5849d98640 1 -- 172.21.15.26:0/1986086939 shutdown_connections 2024-04-21T10:30:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.713+0000 7f5849d98640 1 -- 172.21.15.26:0/1986086939 wait complete. 2024-04-21T10:30:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.713+0000 7f5849d98640 1 Processor -- start 2024-04-21T10:30:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5849d98640 1 -- start start 2024-04-21T10:30:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5849d98640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5844102ad0 0x7f58441981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5849d98640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5844105400 0x7f5844198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5849d98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5844107d30 0x7f584419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5849d98640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f584410db10 con 0x7f5844107d30 2024-04-21T10:30:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5849d98640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f584410d990 con 0x7f5844102ad0 2024-04-21T10:30:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5849d98640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f584410dc90 con 0x7f5844105400 2024-04-21T10:30:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5843fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5844107d30 0x7f584419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5843fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5844107d30 0x7f584419f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47946/0 (socket says 172.21.15.26:47946) 2024-04-21T10:30:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5843fff640 1 -- 172.21.15.26:0/2016984634 learned_addr learned my addr 172.21.15.26:0/2016984634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5842ffd640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5844105400 0x7f5844198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f58437fe640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5844102ad0 0x7f58441981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5843fff640 1 -- 172.21.15.26:0/2016984634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5844105400 msgr2=0x7f5844198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5843fff640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5844105400 0x7f5844198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5843fff640 1 -- 172.21.15.26:0/2016984634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5844102ad0 msgr2=0x7f58441981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5843fff640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5844102ad0 0x7f58441981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5843fff640 1 -- 172.21.15.26:0/2016984634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584419fe10 con 0x7f5844107d30 2024-04-21T10:30:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5843fff640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5844107d30 0x7f584419f7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f583c00cce0 tx=0x7f583c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5840ff9640 1 -- 172.21.15.26:0/2016984634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c00f450 con 0x7f5844107d30 2024-04-21T10:30:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5849d98640 1 -- 172.21.15.26:0/2016984634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58441a0100 con 0x7f5844107d30 2024-04-21T10:30:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5840ff9640 1 -- 172.21.15.26:0/2016984634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f583c00faa0 con 0x7f5844107d30 2024-04-21T10:30:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5840ff9640 1 -- 172.21.15.26:0/2016984634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c018b70 con 0x7f5844107d30 2024-04-21T10:30:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.714+0000 7f5849d98640 1 -- 172.21.15.26:0/2016984634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58441a0650 con 0x7f5844107d30 2024-04-21T10:30:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.715+0000 7f5849d98640 1 -- 172.21.15.26:0/2016984634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58440fdf30 con 0x7f5844107d30 2024-04-21T10:30:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.717+0000 7f5840ff9640 1 -- 172.21.15.26:0/2016984634 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f583c018d10 con 0x7f5844107d30 2024-04-21T10:30:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.717+0000 7f5840ff9640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f582c0a3c40 0x7f582c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.717+0000 7f58437fe640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f582c0a3c40 0x7f582c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.717+0000 7f5840ff9640 1 -- 172.21.15.26:0/2016984634 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f583c0d0b50 con 0x7f5844107d30 2024-04-21T10:30:23.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.718+0000 7f58437fe640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f582c0a3c40 0x7f582c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f58340096c0 tx=0x7f5834000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:23.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.721+0000 7f5840ff9640 1 -- 172.21.15.26:0/2016984634 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f583c014030 con 0x7f5844107d30 2024-04-21T10:30:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:23 smithi190 ceph-mon[31423]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:30:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:23 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2172945994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:30:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:23 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2840251133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:23 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/938265962' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.907+0000 7f5849d98640 1 -- 172.21.15.26:0/2016984634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5844062b50 con 0x7f5844107d30 2024-04-21T10:30:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.907+0000 7f5840ff9640 1 -- 172.21.15.26:0/2016984634 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f583c09d450 con 0x7f5844107d30 2024-04-21T10:30:23.907 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-21T10:30:23.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.910+0000 7f5849d98640 1 -- 172.21.15.26:0/2016984634 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f582c0a3c40 msgr2=0x7f582c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.910+0000 7f5849d98640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f582c0a3c40 0x7f582c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f58340096c0 tx=0x7f5834000bc0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.910+0000 7f5849d98640 1 -- 172.21.15.26:0/2016984634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5844107d30 msgr2=0x7f584419f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.910+0000 7f5849d98640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5844107d30 0x7f584419f7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f583c00cce0 tx=0x7f583c00a620 comp rx=0 tx=0).stop 2024-04-21T10:30:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.910+0000 7f5849d98640 1 -- 172.21.15.26:0/2016984634 shutdown_connections 2024-04-21T10:30:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.910+0000 7f5849d98640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f582c0a3c40 0x7f582c0a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.910+0000 7f5849d98640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5844107d30 0x7f584419f7a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.910+0000 7f5849d98640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5844105400 0x7f5844198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.910+0000 7f5849d98640 1 --2- 172.21.15.26:0/2016984634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5844102ad0 0x7f58441981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.910+0000 7f5849d98640 1 -- 172.21.15.26:0/2016984634 >> 172.21.15.26:0/2016984634 conn(0x7f58440fc440 msgr2=0x7f5844103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.910+0000 7f5849d98640 1 -- 172.21.15.26:0/2016984634 shutdown_connections 2024-04-21T10:30:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:23.910+0000 7f5849d98640 1 -- 172.21.15.26:0/2016984634 wait complete. 2024-04-21T10:30:23.921 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T10:30:24.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.026+0000 7f0602877640 1 -- 172.21.15.26:0/1717013680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc100e60 msgr2=0x7f05fc101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.026+0000 7f0602877640 1 --2- 172.21.15.26:0/1717013680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc100e60 0x7f05fc101280 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009900 tx=0x7f05ec02bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.026+0000 7f0602877640 1 -- 172.21.15.26:0/1717013680 shutdown_connections 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.026+0000 7f0602877640 1 --2- 172.21.15.26:0/1717013680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05fc102a00 0x7f05fc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.026+0000 7f0602877640 1 --2- 172.21.15.26:0/1717013680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc102040 0x7f05fc1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.026+0000 7f0602877640 1 --2- 172.21.15.26:0/1717013680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc100e60 0x7f05fc101280 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.026+0000 7f0602877640 1 -- 172.21.15.26:0/1717013680 >> 172.21.15.26:0/1717013680 conn(0x7f05fc0fc440 msgr2=0x7f05fc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.026+0000 7f0602877640 1 -- 172.21.15.26:0/1717013680 shutdown_connections 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.026+0000 7f0602877640 1 -- 172.21.15.26:0/1717013680 wait complete. 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0602877640 1 Processor -- start 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0602877640 1 -- start start 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0602877640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05fc100e60 0x7f05fc06b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0602877640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc102040 0x7f05fc19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0602877640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc102a00 0x7f05fc19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0602877640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc10c9f0 con 0x7f05fc102a00 2024-04-21T10:30:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0602877640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc10c870 con 0x7f05fc100e60 2024-04-21T10:30:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0602877640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc10cb70 con 0x7f05fc102040 2024-04-21T10:30:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0600e14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc102a00 0x7f05fc19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0600e14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc102a00 0x7f05fc19f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47962/0 (socket says 172.21.15.26:47962) 2024-04-21T10:30:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0600e14640 1 -- 172.21.15.26:0/3641915225 learned_addr learned my addr 172.21.15.26:0/3641915225 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f05fbfff640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05fc100e60 0x7f05fc06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f05fb7fe640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc102040 0x7f05fc19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0600e14640 1 -- 172.21.15.26:0/3641915225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc102040 msgr2=0x7f05fc19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.027+0000 7f0600e14640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc102040 0x7f05fc19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.028+0000 7f0600e14640 1 -- 172.21.15.26:0/3641915225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05fc100e60 msgr2=0x7f05fc06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.028+0000 7f0600e14640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05fc100e60 0x7f05fc06b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.028+0000 7f0600e14640 1 -- 172.21.15.26:0/3641915225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc19fe60 con 0x7f05fc102a00 2024-04-21T10:30:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.028+0000 7f0600e14640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc102a00 0x7f05fc19f5a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009fd0 tx=0x7f05f400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.028+0000 7f05f97fa640 1 -- 172.21.15.26:0/3641915225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4017070 con 0x7f05fc102a00 2024-04-21T10:30:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.028+0000 7f05f97fa640 1 -- 172.21.15.26:0/3641915225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05f4003e20 con 0x7f05fc102a00 2024-04-21T10:30:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.028+0000 7f0602877640 1 -- 172.21.15.26:0/3641915225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc1a0150 con 0x7f05fc102a00 2024-04-21T10:30:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.028+0000 7f05f97fa640 1 -- 172.21.15.26:0/3641915225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4010460 con 0x7f05fc102a00 2024-04-21T10:30:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.028+0000 7f0602877640 1 -- 172.21.15.26:0/3641915225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc1a0650 con 0x7f05fc102a00 2024-04-21T10:30:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.029+0000 7f0602877640 1 -- 172.21.15.26:0/3641915225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc1082c0 con 0x7f05fc102a00 2024-04-21T10:30:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.030+0000 7f05f97fa640 1 -- 172.21.15.26:0/3641915225 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f05f4003970 con 0x7f05fc102a00 2024-04-21T10:30:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.031+0000 7f05f97fa640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f05e40a3c40 0x7f05e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.031+0000 7f05fbfff640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f05e40a3c40 0x7f05e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.031+0000 7f05f97fa640 1 -- 172.21.15.26:0/3641915225 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f05f40d03c0 con 0x7f05fc102a00 2024-04-21T10:30:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.031+0000 7f05fbfff640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f05e40a3c40 0x7f05e40a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f05ec005ab0 tx=0x7f05ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:24.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.035+0000 7f05f97fa640 1 -- 172.21.15.26:0/3641915225 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05f409cbd0 con 0x7f05fc102a00 2024-04-21T10:30:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.221+0000 7f0602877640 1 -- 172.21.15.26:0/3641915225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f05fc062b50 con 0x7f05fc102a00 2024-04-21T10:30:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.221+0000 7f05f97fa640 1 -- 172.21.15.26:0/3641915225 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f05f409c4d0 con 0x7f05fc102a00 2024-04-21T10:30:24.221 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-21T10:30:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.223+0000 7f0602877640 1 -- 172.21.15.26:0/3641915225 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f05e40a3c40 msgr2=0x7f05e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.223+0000 7f0602877640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f05e40a3c40 0x7f05e40a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f05ec005ab0 tx=0x7f05ec005fd0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.223+0000 7f0602877640 1 -- 172.21.15.26:0/3641915225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc102a00 msgr2=0x7f05fc19f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.223+0000 7f0602877640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc102a00 0x7f05fc19f5a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009fd0 tx=0x7f05f400b260 comp rx=0 tx=0).stop 2024-04-21T10:30:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.223+0000 7f0602877640 1 -- 172.21.15.26:0/3641915225 shutdown_connections 2024-04-21T10:30:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.223+0000 7f0602877640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f05e40a3c40 0x7f05e40a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.223+0000 7f0602877640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc102a00 0x7f05fc19f5a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.223+0000 7f0602877640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05fc102040 0x7f05fc19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.223+0000 7f0602877640 1 --2- 172.21.15.26:0/3641915225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05fc100e60 0x7f05fc06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.223+0000 7f0602877640 1 -- 172.21.15.26:0/3641915225 >> 172.21.15.26:0/3641915225 conn(0x7f05fc0fc440 msgr2=0x7f05fc105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.223+0000 7f0602877640 1 -- 172.21.15.26:0/3641915225 shutdown_connections 2024-04-21T10:30:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.223+0000 7f0602877640 1 -- 172.21.15.26:0/3641915225 wait complete. 2024-04-21T10:30:24.234 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:30:24.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.336+0000 7f2949d32640 1 -- 172.21.15.26:0/2227094813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29441029f0 msgr2=0x7f2944104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.336+0000 7f2949d32640 1 --2- 172.21.15.26:0/2227094813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29441029f0 0x7f2944104de0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2934009900 tx=0x7f293402bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:24.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.336+0000 7f2949d32640 1 -- 172.21.15.26:0/2227094813 shutdown_connections 2024-04-21T10:30:24.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.336+0000 7f2949d32640 1 --2- 172.21.15.26:0/2227094813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2944107c50 0x7f294410a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.336+0000 7f2949d32640 1 --2- 172.21.15.26:0/2227094813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2944105320 0x7f2944107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.336+0000 7f2949d32640 1 --2- 172.21.15.26:0/2227094813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29441029f0 0x7f2944104de0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.336+0000 7f2949d32640 1 -- 172.21.15.26:0/2227094813 >> 172.21.15.26:0/2227094813 conn(0x7f29440fc420 msgr2=0x7f29440fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.336+0000 7f2949d32640 1 -- 172.21.15.26:0/2227094813 shutdown_connections 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.336+0000 7f2949d32640 1 -- 172.21.15.26:0/2227094813 wait complete. 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.337+0000 7f2949d32640 1 Processor -- start 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.337+0000 7f2949d32640 1 -- start start 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.337+0000 7f2949d32640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29441029f0 0x7f29441981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.337+0000 7f2949d32640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2944105320 0x7f2944198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.337+0000 7f2949d32640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2944107c50 0x7f294419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.337+0000 7f2949d32640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f294410dab0 con 0x7f2944105320 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.337+0000 7f2949d32640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f294410d930 con 0x7f2944107c50 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.337+0000 7f2949d32640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f294410dc30 con 0x7f29441029f0 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.337+0000 7f2942ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2944105320 0x7f2944198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.337+0000 7f2942ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2944105320 0x7f2944198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47976/0 (socket says 172.21.15.26:47976) 2024-04-21T10:30:24.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.337+0000 7f2942ffd640 1 -- 172.21.15.26:0/1112912556 learned_addr learned my addr 172.21.15.26:0/1112912556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.337+0000 7f2943fff640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2944107c50 0x7f294419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.338+0000 7f29437fe640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29441029f0 0x7f29441981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.338+0000 7f2942ffd640 1 -- 172.21.15.26:0/1112912556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29441029f0 msgr2=0x7f29441981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.338+0000 7f2942ffd640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29441029f0 0x7f29441981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.338+0000 7f2942ffd640 1 -- 172.21.15.26:0/1112912556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2944107c50 msgr2=0x7f294419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.338+0000 7f2942ffd640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2944107c50 0x7f294419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.338+0000 7f2942ffd640 1 -- 172.21.15.26:0/1112912556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294419fe00 con 0x7f2944105320 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.338+0000 7f2942ffd640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2944105320 0x7f2944198720 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f293800d740 tx=0x7f293800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.338+0000 7f2940ff9640 1 -- 172.21.15.26:0/1112912556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938017070 con 0x7f2944105320 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.338+0000 7f2949d32640 1 -- 172.21.15.26:0/1112912556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29441a00f0 con 0x7f2944105320 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.338+0000 7f2940ff9640 1 -- 172.21.15.26:0/1112912556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293800de50 con 0x7f2944105320 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.338+0000 7f2940ff9640 1 -- 172.21.15.26:0/1112912556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938004ce0 con 0x7f2944105320 2024-04-21T10:30:24.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.338+0000 7f2949d32640 1 -- 172.21.15.26:0/1112912556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29441a05f0 con 0x7f2944105320 2024-04-21T10:30:24.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.339+0000 7f2949d32640 1 -- 172.21.15.26:0/1112912556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29440fdf10 con 0x7f2944105320 2024-04-21T10:30:24.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.340+0000 7f2940ff9640 1 -- 172.21.15.26:0/1112912556 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2938004e80 con 0x7f2944105320 2024-04-21T10:30:24.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.341+0000 7f2940ff9640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f292c0a3c40 0x7f292c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:24.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.341+0000 7f29437fe640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f292c0a3c40 0x7f292c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:24.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.341+0000 7f2940ff9640 1 -- 172.21.15.26:0/1112912556 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f29380d03c0 con 0x7f2944105320 2024-04-21T10:30:24.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.341+0000 7f29437fe640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f292c0a3c40 0x7f292c0a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2934000c00 tx=0x7f2934005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:24.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.345+0000 7f2940ff9640 1 -- 172.21.15.26:0/1112912556 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f293809cbd0 con 0x7f2944105320 2024-04-21T10:30:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.570+0000 7f2949d32640 1 -- 172.21.15.26:0/1112912556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f294402d5c0 con 0x7f2944105320 2024-04-21T10:30:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.571+0000 7f2940ff9640 1 -- 172.21.15.26:0/1112912556 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11884 (secure 0 0 0) 0x7f293809c4d0 con 0x7f2944105320 2024-04-21T10:30:24.571 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:24.571 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:29:05.806617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24478,24488,24496]},"id":1}]} 2024-04-21T10:30:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 20 2024-04-21T10:30:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.573+0000 7f2949d32640 1 -- 172.21.15.26:0/1112912556 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f292c0a3c40 msgr2=0x7f292c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.573+0000 7f2949d32640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f292c0a3c40 0x7f292c0a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2934000c00 tx=0x7f2934005be0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.573+0000 7f2949d32640 1 -- 172.21.15.26:0/1112912556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2944105320 msgr2=0x7f2944198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.573+0000 7f2949d32640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2944105320 0x7f2944198720 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f293800d740 tx=0x7f293800dc00 comp rx=0 tx=0).stop 2024-04-21T10:30:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.573+0000 7f2949d32640 1 -- 172.21.15.26:0/1112912556 shutdown_connections 2024-04-21T10:30:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.573+0000 7f2949d32640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f292c0a3c40 0x7f292c0a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.573+0000 7f2949d32640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2944107c50 0x7f294419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.573+0000 7f2949d32640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2944105320 0x7f2944198720 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.573+0000 7f2949d32640 1 --2- 172.21.15.26:0/1112912556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29441029f0 0x7f29441981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.573+0000 7f2949d32640 1 -- 172.21.15.26:0/1112912556 >> 172.21.15.26:0/1112912556 conn(0x7f29440fc420 msgr2=0x7f29441034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.573+0000 7f2949d32640 1 -- 172.21.15.26:0/1112912556 shutdown_connections 2024-04-21T10:30:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.573+0000 7f2949d32640 1 -- 172.21.15.26:0/1112912556 wait complete. 2024-04-21T10:30:24.583 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:24 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3394566064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:30:24.583 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:24 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2016984634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:30:24.583 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:24 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3641915225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:30:24.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:30:24.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.686+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3697743648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb288103190 msgr2=0x7fb288103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.686+0000 7fb28d9f4640 1 --2- 172.21.15.26:0/3697743648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb288103190 0x7fb288103610 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb270009900 tx=0x7fb27002bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:24.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.686+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3697743648 shutdown_connections 2024-04-21T10:30:24.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.686+0000 7fb28d9f4640 1 --2- 172.21.15.26:0/3697743648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb288103b50 0x7fb28810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.686+0000 7fb28d9f4640 1 --2- 172.21.15.26:0/3697743648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb288103190 0x7fb288103610 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.686+0000 7fb28d9f4640 1 --2- 172.21.15.26:0/3697743648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb288101fa0 0x7fb2881023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.686+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3697743648 >> 172.21.15.26:0/3697743648 conn(0x7fb2880fd530 msgr2=0x7fb2880ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:24.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.686+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3697743648 shutdown_connections 2024-04-21T10:30:24.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.686+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3697743648 wait complete. 2024-04-21T10:30:24.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb28d9f4640 1 Processor -- start 2024-04-21T10:30:24.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb28d9f4640 1 -- start start 2024-04-21T10:30:24.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb28d9f4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb288101fa0 0x7fb288198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:24.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb28d9f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb288103190 0x7fb2881986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb28d9f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb288103b50 0x7fb28819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb28d9f4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb28810dae0 con 0x7fb288103b50 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb28d9f4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb28810d960 con 0x7fb288101fa0 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb28d9f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb28810dc60 con 0x7fb288103190 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb2877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb288103b50 0x7fb28819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb2877fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb288103b50 0x7fb28819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52814/0 (socket says 172.21.15.26:52814) 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb2877fe640 1 -- 172.21.15.26:0/3013686111 learned_addr learned my addr 172.21.15.26:0/3013686111 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb27ffff640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb288103190 0x7fb2881986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb286ffd640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb288101fa0 0x7fb288198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb2877fe640 1 -- 172.21.15.26:0/3013686111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb288103190 msgr2=0x7fb2881986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.687+0000 7fb2877fe640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb288103190 0x7fb2881986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.688+0000 7fb2877fe640 1 -- 172.21.15.26:0/3013686111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb288101fa0 msgr2=0x7fb288198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.688+0000 7fb2877fe640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb288101fa0 0x7fb288198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.688+0000 7fb2877fe640 1 -- 172.21.15.26:0/3013686111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28819fd90 con 0x7fb288103b50 2024-04-21T10:30:24.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.688+0000 7fb2877fe640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb288103b50 0x7fb28819f720 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb28000eaa0 tx=0x7fb28000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:24.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.688+0000 7fb28c9f2640 1 -- 172.21.15.26:0/3013686111 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28000cdb0 con 0x7fb288103b50 2024-04-21T10:30:24.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.688+0000 7fb28c9f2640 1 -- 172.21.15.26:0/3013686111 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb280005020 con 0x7fb288103b50 2024-04-21T10:30:24.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.688+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3013686111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2881a0080 con 0x7fb288103b50 2024-04-21T10:30:24.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.688+0000 7fb28c9f2640 1 -- 172.21.15.26:0/3013686111 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2800116d0 con 0x7fb288103b50 2024-04-21T10:30:24.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.688+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3013686111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2881a0580 con 0x7fb288103b50 2024-04-21T10:30:24.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.689+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3013686111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb288109340 con 0x7fb288103b50 2024-04-21T10:30:24.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.690+0000 7fb28c9f2640 1 -- 172.21.15.26:0/3013686111 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb2800051d0 con 0x7fb288103b50 2024-04-21T10:30:24.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.691+0000 7fb28c9f2640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb2740a3c40 0x7fb2740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:24.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.691+0000 7fb286ffd640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb2740a3c40 0x7fb2740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:24.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.691+0000 7fb28c9f2640 1 -- 172.21.15.26:0/3013686111 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fb2800d08b0 con 0x7fb288103b50 2024-04-21T10:30:24.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.691+0000 7fb286ffd640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb2740a3c40 0x7fb2740a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb278005fd0 tx=0x7fb278005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:24.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.695+0000 7fb28c9f2640 1 -- 172.21.15.26:0/3013686111 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb280016030 con 0x7fb288103b50 2024-04-21T10:30:24.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:24 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3394566064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:30:24.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:24 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2016984634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:30:24.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:24 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3641915225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:30:24.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:24 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3394566064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:30:24.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:24 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2016984634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:30:24.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:24 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3641915225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:30:24.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.887+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3013686111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb288062b50 con 0x7fb288103b50 2024-04-21T10:30:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.888+0000 7fb28c9f2640 1 -- 172.21.15.26:0/3013686111 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22705 (secure 0 0 0) 0x7fb28009d0c0 con 0x7fb288103b50 2024-04-21T10:30:24.888 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:24.888 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:30:03.297428+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:30:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.891+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3013686111 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb2740a3c40 msgr2=0x7fb2740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.891+0000 7fb28d9f4640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb2740a3c40 0x7fb2740a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb278005fd0 tx=0x7fb278005850 comp rx=0 tx=0).stop 2024-04-21T10:30:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.891+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3013686111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb288103b50 msgr2=0x7fb28819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.891+0000 7fb28d9f4640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb288103b50 0x7fb28819f720 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb28000eaa0 tx=0x7fb28000ef60 comp rx=0 tx=0).stop 2024-04-21T10:30:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.891+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3013686111 shutdown_connections 2024-04-21T10:30:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.891+0000 7fb28d9f4640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb2740a3c40 0x7fb2740a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.891+0000 7fb28d9f4640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb288103b50 0x7fb28819f720 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.891+0000 7fb28d9f4640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb288103190 0x7fb2881986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.891+0000 7fb28d9f4640 1 --2- 172.21.15.26:0/3013686111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb288101fa0 0x7fb288198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.891+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3013686111 >> 172.21.15.26:0/3013686111 conn(0x7fb2880fd530 msgr2=0x7fb288106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.891+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3013686111 shutdown_connections 2024-04-21T10:30:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:24.891+0000 7fb28d9f4640 1 -- 172.21.15.26:0/3013686111 wait complete. 2024-04-21T10:30:24.902 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:30:25.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.004+0000 7f9f2e055640 1 -- 172.21.15.26:0/4119429621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f28103b40 msgr2=0x7f9f2810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.004+0000 7f9f2e055640 1 --2- 172.21.15.26:0/4119429621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f28103b40 0x7f9f2810a3d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009ca0 tx=0x7f9f2002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.004+0000 7f9f2e055640 1 -- 172.21.15.26:0/4119429621 shutdown_connections 2024-04-21T10:30:25.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.004+0000 7f9f2e055640 1 --2- 172.21.15.26:0/4119429621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f28103b40 0x7f9f2810a3d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.004+0000 7f9f2e055640 1 --2- 172.21.15.26:0/4119429621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f28103180 0x7f9f28103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.004+0000 7f9f2e055640 1 --2- 172.21.15.26:0/4119429621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f28101f90 0x7f9f281023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.004+0000 7f9f2e055640 1 -- 172.21.15.26:0/4119429621 >> 172.21.15.26:0/4119429621 conn(0x7f9f280fd500 msgr2=0x7f9f280ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.004+0000 7f9f2e055640 1 -- 172.21.15.26:0/4119429621 shutdown_connections 2024-04-21T10:30:25.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.004+0000 7f9f2e055640 1 -- 172.21.15.26:0/4119429621 wait complete. 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f2e055640 1 Processor -- start 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f2e055640 1 -- start start 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f2e055640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f28101f90 0x7f9f281980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f2e055640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f28103180 0x7f9f281985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f2e055640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f28103b40 0x7f9f2819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f2e055640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2810da90 con 0x7f9f28103180 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f2e055640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2810d910 con 0x7f9f28101f90 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f2e055640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2810dc10 con 0x7f9f28103b40 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f26ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f28103180 0x7f9f281985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f26ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f28103180 0x7f9f281985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52832/0 (socket says 172.21.15.26:52832) 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f26ffd640 1 -- 172.21.15.26:0/1602519648 learned_addr learned my addr 172.21.15.26:0/1602519648 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f277fe640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f28101f90 0x7f9f281980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f27fff640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f28103b40 0x7f9f2819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f26ffd640 1 -- 172.21.15.26:0/1602519648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f28103b40 msgr2=0x7f9f2819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f26ffd640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f28103b40 0x7f9f2819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f26ffd640 1 -- 172.21.15.26:0/1602519648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f28101f90 msgr2=0x7f9f281980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f26ffd640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f28101f90 0x7f9f281980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.005+0000 7f9f26ffd640 1 -- 172.21.15.26:0/1602519648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2819fcc0 con 0x7f9f28103180 2024-04-21T10:30:25.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.006+0000 7f9f26ffd640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f28103180 0x7f9f281985e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00c890 tx=0x7f9f1c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.006+0000 7f9f24ff9640 1 -- 172.21.15.26:0/1602519648 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c004220 con 0x7f9f28103180 2024-04-21T10:30:25.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.006+0000 7f9f24ff9640 1 -- 172.21.15.26:0/1602519648 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f1c0043c0 con 0x7f9f28103180 2024-04-21T10:30:25.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.006+0000 7f9f24ff9640 1 -- 172.21.15.26:0/1602519648 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c004a00 con 0x7f9f28103180 2024-04-21T10:30:25.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.006+0000 7f9f2e055640 1 -- 172.21.15.26:0/1602519648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2819ffb0 con 0x7f9f28103180 2024-04-21T10:30:25.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.006+0000 7f9f2e055640 1 -- 172.21.15.26:0/1602519648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f281a04c0 con 0x7f9f28103180 2024-04-21T10:30:25.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.007+0000 7f9f2e055640 1 -- 172.21.15.26:0/1602519648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f28109250 con 0x7f9f28103180 2024-04-21T10:30:25.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.008+0000 7f9f24ff9640 1 -- 172.21.15.26:0/1602519648 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9f1c0079b0 con 0x7f9f28103180 2024-04-21T10:30:25.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.009+0000 7f9f24ff9640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f9f100a3cc0 0x7f9f100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.009+0000 7f9f277fe640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f9f100a3cc0 0x7f9f100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.009+0000 7f9f24ff9640 1 -- 172.21.15.26:0/1602519648 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f9f1c0d0570 con 0x7f9f28103180 2024-04-21T10:30:25.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.009+0000 7f9f277fe640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f9f100a3cc0 0x7f9f100a6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f28102fe0 tx=0x7f9f18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.013+0000 7f9f24ff9640 1 -- 172.21.15.26:0/1602519648 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f1c014030 con 0x7f9f28103180 2024-04-21T10:30:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.238+0000 7f9f2e055640 1 -- 172.21.15.26:0/1602519648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f2802d810 con 0x7f9f28103180 2024-04-21T10:30:25.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.238+0000 7f9f24ff9640 1 -- 172.21.15.26:0/1602519648 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11884 (secure 0 0 0) 0x7f9f1c09c810 con 0x7f9f28103180 2024-04-21T10:30:25.238 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:25.238 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:29:05.806617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24478,24488,24496]},"id":1}]} 2024-04-21T10:30:25.238 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 20 2024-04-21T10:30:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.240+0000 7f9f2e055640 1 -- 172.21.15.26:0/1602519648 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f9f100a3cc0 msgr2=0x7f9f100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.240+0000 7f9f2e055640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f9f100a3cc0 0x7f9f100a6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f28102fe0 tx=0x7f9f18005f90 comp rx=0 tx=0).stop 2024-04-21T10:30:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.240+0000 7f9f2e055640 1 -- 172.21.15.26:0/1602519648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f28103180 msgr2=0x7f9f281985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.240+0000 7f9f2e055640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f28103180 0x7f9f281985e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00c890 tx=0x7f9f1c00cd50 comp rx=0 tx=0).stop 2024-04-21T10:30:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.240+0000 7f9f2e055640 1 -- 172.21.15.26:0/1602519648 shutdown_connections 2024-04-21T10:30:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.240+0000 7f9f2e055640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f9f100a3cc0 0x7f9f100a6180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.240+0000 7f9f2e055640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f28103b40 0x7f9f2819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.240+0000 7f9f2e055640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f28103180 0x7f9f281985e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.240+0000 7f9f2e055640 1 --2- 172.21.15.26:0/1602519648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f28101f90 0x7f9f281980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.240+0000 7f9f2e055640 1 -- 172.21.15.26:0/1602519648 >> 172.21.15.26:0/1602519648 conn(0x7f9f280fd500 msgr2=0x7f9f28106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.240+0000 7f9f2e055640 1 -- 172.21.15.26:0/1602519648 shutdown_connections 2024-04-21T10:30:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.240+0000 7f9f2e055640 1 -- 172.21.15.26:0/1602519648 wait complete. 2024-04-21T10:30:25.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T10:29:01.278914+0000', 'modified': '2024-04-21T10:29:05.806617+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24496, 'mds_1': 24488, 'mds_2': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6835/1594119566', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 1594119566}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 1594119566}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6837/2325983283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 2325983283}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 2325983283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6835/1573483809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 1573483809}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 1573483809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.190:6837/2091448511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 2091448511}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 2091448511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 1, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6839/2428626465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 2428626465}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 2428626465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.190:6839/1574521545', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 1574521545}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 1574521545}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24496, 'qdb_cluster': [24478, 24488, 24496]} 2024-04-21T10:30:25.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-21T10:30:25.252 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-21T10:30:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.355+0000 7f7df52d2640 1 -- 172.21.15.26:0/351800467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df0101fb0 msgr2=0x7f7df01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.355+0000 7f7df52d2640 1 --2- 172.21.15.26:0/351800467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df0101fb0 0x7f7df01023d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009900 tx=0x7f7de002bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.355+0000 7f7df52d2640 1 -- 172.21.15.26:0/351800467 shutdown_connections 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.355+0000 7f7df52d2640 1 --2- 172.21.15.26:0/351800467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df0103b60 0x7f7df010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.355+0000 7f7df52d2640 1 --2- 172.21.15.26:0/351800467 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7df01031a0 0x7f7df0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.355+0000 7f7df52d2640 1 --2- 172.21.15.26:0/351800467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df0101fb0 0x7f7df01023d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.355+0000 7f7df52d2640 1 -- 172.21.15.26:0/351800467 >> 172.21.15.26:0/351800467 conn(0x7f7df00fd520 msgr2=0x7f7df00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.355+0000 7f7df52d2640 1 -- 172.21.15.26:0/351800467 shutdown_connections 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.355+0000 7f7df52d2640 1 -- 172.21.15.26:0/351800467 wait complete. 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.355+0000 7f7df52d2640 1 Processor -- start 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.355+0000 7f7df52d2640 1 -- start start 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7df52d2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df0101fb0 0x7f7df0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7df52d2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7df01031a0 0x7f7df01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7df52d2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df0103b60 0x7f7df019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7df52d2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7df010db30 con 0x7f7df0101fb0 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7df52d2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7df010d9b0 con 0x7f7df0103b60 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7df52d2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7df010dcb0 con 0x7f7df01031a0 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7deeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df0101fb0 0x7f7df0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7deeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df0101fb0 0x7f7df0198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52860/0 (socket says 172.21.15.26:52860) 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7deeffd640 1 -- 172.21.15.26:0/3516340506 learned_addr learned my addr 172.21.15.26:0/3516340506 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7deeffd640 1 -- 172.21.15.26:0/3516340506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7df01031a0 msgr2=0x7f7df01987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7def7fe640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df0103b60 0x7f7df019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7dee7fc640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7df01031a0 0x7f7df01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7deeffd640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7df01031a0 0x7f7df01987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7deeffd640 1 -- 172.21.15.26:0/3516340506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df0103b60 msgr2=0x7f7df019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7deeffd640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df0103b60 0x7f7df019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7deeffd640 1 -- 172.21.15.26:0/3516340506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df019feb0 con 0x7f7df0101fb0 2024-04-21T10:30:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7def7fe640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df0103b60 0x7f7df019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7deeffd640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df0101fb0 0x7f7df0198290 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7de0000c00 tx=0x7f7de0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7dd7fff640 1 -- 172.21.15.26:0/3516340506 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0043020 con 0x7f7df0101fb0 2024-04-21T10:30:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7dd7fff640 1 -- 172.21.15.26:0/3516340506 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7de0004aa0 con 0x7f7df0101fb0 2024-04-21T10:30:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7df52d2640 1 -- 172.21.15.26:0/3516340506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df01a0140 con 0x7f7df0101fb0 2024-04-21T10:30:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7dd7fff640 1 -- 172.21.15.26:0/3516340506 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0003d00 con 0x7f7df0101fb0 2024-04-21T10:30:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.356+0000 7f7df52d2640 1 -- 172.21.15.26:0/3516340506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df01a0710 con 0x7f7df0101fb0 2024-04-21T10:30:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.358+0000 7f7dd7fff640 1 -- 172.21.15.26:0/3516340506 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7de0046020 con 0x7f7df0101fb0 2024-04-21T10:30:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7df7fa5740 1 Processor -- start 2024-04-21T10:30:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7df7fa5740 1 -- start start 2024-04-21T10:30:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7df7fa5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 0x560150c7c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7df7fa5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c7d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7df7fa5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 0x560150c842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7df7fa5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560150bdcb10 con 0x560150c7cc40 2024-04-21T10:30:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7df7fa5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560150b7a240 con 0x560150c7c250 2024-04-21T10:30:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7df7fa5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560150bd8ad0 con 0x560150c7d6e0 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7dd7fff640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7dd80a3c40 0x7f7dd80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7deeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c7d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7deeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c7d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52868/0 (socket says 172.21.15.26:52868) 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7deeffd640 1 -- 172.21.15.26:0/2771020890 learned_addr learned my addr 172.21.15.26:0/2771020890 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7def7fe640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7dd80a3c40 0x7f7dd80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7def7fe640 1 --2- 172.21.15.26:0/2771020890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 0x560150c7c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7dee7fc640 1 --2- 172.21.15.26:0/2771020890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 0x560150c842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.359+0000 7f7dd7fff640 1 -- 172.21.15.26:0/3516340506 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f7de00f2ad0 con 0x7f7df0101fb0 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.360+0000 7f7deeffd640 1 -- 172.21.15.26:0/2771020890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 msgr2=0x560150c842c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.360+0000 7f7deeffd640 1 --2- 172.21.15.26:0/2771020890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 0x560150c842c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.360+0000 7f7deeffd640 1 -- 172.21.15.26:0/2771020890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 msgr2=0x560150c7c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.360+0000 7f7deeffd640 1 --2- 172.21.15.26:0/2771020890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 0x560150c7c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.360+0000 7f7deeffd640 1 -- 172.21.15.26:0/2771020890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601508ef2d0 con 0x560150c7cc40 2024-04-21T10:30:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.360+0000 7f7def7fe640 1 --2- 172.21.15.26:0/2771020890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 0x560150c7c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.360+0000 7f7def7fe640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7dd80a3c40 0x7f7dd80a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7de8005c40 tx=0x7f7de8013a60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.360+0000 7f7deeffd640 1 --2- 172.21.15.26:0/2771020890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c7d0c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7de004e000 tx=0x7f7de004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ed0780e35ea78c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.360+0000 7f7dd57fa640 1 -- 172.21.15.26:0/2771020890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de00bf450 con 0x560150c7cc40 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.360+0000 7f7dd57fa640 1 -- 172.21.15.26:0/2771020890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7de00bed50 con 0x560150c7cc40 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.360+0000 7f7dd57fa640 1 -- 172.21.15.26:0/2771020890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de00c7530 con 0x560150c7cc40 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 -- 172.21.15.26:0/2771020890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 msgr2=0x560150c7d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/2771020890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c7d0c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7de004e000 tx=0x7f7de004f000 comp rx=0 tx=0).stop 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 -- 172.21.15.26:0/2771020890 shutdown_connections 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/2771020890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 0x560150c842c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/2771020890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c7d0c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/2771020890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 0x560150c7c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 -- 172.21.15.26:0/2771020890 >> 172.21.15.26:0/2771020890 conn(0x560150c73f40 msgr2=0x560150c72490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 -- 172.21.15.26:0/2771020890 shutdown_connections 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 -- 172.21.15.26:0/2771020890 wait complete. 2024-04-21T10:30:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 -- start start 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 Processor -- start 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 0x560150c86020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c86560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 0x560150c8d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560150c7dee0 con 0x560150c7cc40 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560150b65190 con 0x560150c7c250 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560150c84800 con 0x560150c7d6e0 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7df7fa5740 10 client.0 _ll_register_callbacks cb 0x560150b608b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7dee7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c86560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7dee7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c86560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52872/0 (socket says 172.21.15.26:52872) 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7dee7fc640 1 -- 172.21.15.26:0/1671620043 learned_addr learned my addr 172.21.15.26:0/1671620043 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.361+0000 7f7deeffd640 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 0x560150c86020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7def7fe640 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 0x560150c8d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7dee7fc640 1 -- 172.21.15.26:0/1671620043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 msgr2=0x560150c8d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7dee7fc640 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 0x560150c8d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7dee7fc640 1 -- 172.21.15.26:0/1671620043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 msgr2=0x560150c86020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7dee7fc640 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 0x560150c86020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7dee7fc640 1 -- 172.21.15.26:0/1671620043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601509b96b0 con 0x560150c7cc40 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7dee7fc640 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c86560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7de4006e50 tx=0x7f7de400e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49418ea1b8f37fd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7dd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7dd2ff5640 1 -- 172.21.15.26:0/1671620043 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4018020 con 0x560150c7cc40 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7dd2ff5640 1 -- 172.21.15.26:0/1671620043 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f7de4013660 con 0x560150c7cc40 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7df7fa5740 1 -- 172.21.15.26:0/1671620043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de00bed50 con 0x560150c7cc40 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7dd2ff5640 1 -- 172.21.15.26:0/1671620043 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7de4023050 con 0x560150c7cc40 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7df7fa5740 10 client.15060 fetch_fsmap learned FSMap version 20 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7df7fa5740 1 -- 172.21.15.26:0/1671620043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560150c909e0 con 0x560150c7cc40 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7dd2ff5640 1 -- 172.21.15.26:0/1671620043 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7de00bed50 con 0x560150c7cc40 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.362+0000 7f7dd2ff5640 1 -- 172.21.15.26:0/1671620043 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f7de40148b0 con 0x560150c7cc40 2024-04-21T10:30:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.363+0000 7f7df7fa5740 10 client.15060 fetch_fsmap finished waiting for FSMap version 20 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.363+0000 7f7df7fa5740 10 client.15060 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.363+0000 7f7df7fa5740 20 client.15060 populate_metadata read hostname 'smithi026' 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.363+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x560150c93130 0x560150c945a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.363+0000 7f7df7fa5740 4 client.15060 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.363+0000 7f7df7fa5740 1 -- 172.21.15.26:0/1671620043 --> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560150bea530 con 0x560150c93130 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.363+0000 7f7deeffd640 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x560150c93130 0x560150c945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.363+0000 7f7dd2ff5640 10 client.15060 ms_handle_connect on v2:172.21.15.190:6836/2091448511 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.363+0000 7f7deeffd640 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x560150c93130 0x560150c945a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e869fd48eb4352a3 server_cookie=202fc8edc1a24177 in_seq=0 out_seq=0 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7dd2ff5640 1 -- 172.21.15.26:0/1671620043 <== mds.2 v2:172.21.15.190:6836/2091448511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560150bea530 con 0x560150c93130 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7dd2ff5640 10 client.15060 handle_command_reply: tid=0 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7df7fa5740 1 client.15060 shutdown 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7df7fa5740 1 -- 172.21.15.26:0/1671620043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 msgr2=0x560150c86560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c86560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7de4006e50 tx=0x7f7de400e6e0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7df7fa5740 1 -- 172.21.15.26:0/1671620043 shutdown_connections 2024-04-21T10:30:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 0x560150c8d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x560150c93130 0x560150c945a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c86560 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/1671620043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 0x560150c86020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7dd2ff5640 0 client.15060 ms_handle_reset on v2:172.21.15.190:6836/2091448511 2024-04-21T10:30:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7df7fa5740 1 -- 172.21.15.26:0/1671620043 >> 172.21.15.26:0/1671620043 conn(0x560150c73f40 msgr2=0x560150c71a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.364+0000 7f7df7fa5740 1 -- 172.21.15.26:0/1671620043 shutdown_connections 2024-04-21T10:30:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.365+0000 7f7df7fa5740 1 -- 172.21.15.26:0/1671620043 wait complete. 2024-04-21T10:30:25.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.365+0000 7f7df7fa5740 20 client.15060 trim_cache size 0 max 16384 2024-04-21T10:30:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.382+0000 7f7df7fa5740 1 Processor -- start 2024-04-21T10:30:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.382+0000 7f7df7fa5740 1 -- start start 2024-04-21T10:30:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7df7fa5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 0x560150c802a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7df7fa5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c8d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7df7fa5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 0x560150c80c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7df7fa5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560150c928e0 con 0x560150c7cc40 2024-04-21T10:30:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7df7fa5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560150c92fb0 con 0x560150c7c250 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7df7fa5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560150c93680 con 0x560150c7d6e0 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7dee7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c8d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7dee7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c8d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52882/0 (socket says 172.21.15.26:52882) 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7dee7fc640 1 -- 172.21.15.26:0/3398502103 learned_addr learned my addr 172.21.15.26:0/3398502103 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7dee7fc640 1 -- 172.21.15.26:0/3398502103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 msgr2=0x560150c80c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7dee7fc640 1 --2- 172.21.15.26:0/3398502103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 0x560150c80c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7dee7fc640 1 -- 172.21.15.26:0/3398502103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 msgr2=0x560150c802a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7dee7fc640 1 --2- 172.21.15.26:0/3398502103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 0x560150c802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7dee7fc640 1 -- 172.21.15.26:0/3398502103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560150c85430 con 0x560150c7cc40 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7dee7fc640 1 --2- 172.21.15.26:0/3398502103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c8d560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7de400c970 tx=0x7f7de400c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6acc6ac77a482b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7dd4ff9640 1 -- 172.21.15.26:0/3398502103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4018070 con 0x560150c7cc40 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7dd4ff9640 1 -- 172.21.15.26:0/3398502103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7de4014ab0 con 0x560150c7cc40 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.383+0000 7f7dd4ff9640 1 -- 172.21.15.26:0/3398502103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4026390 con 0x560150c7cc40 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3398502103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 msgr2=0x560150c8d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/3398502103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c8d560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7de400c970 tx=0x7f7de400c340 comp rx=0 tx=0).stop 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3398502103 shutdown_connections 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/3398502103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7d6e0 0x560150c80c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/3398502103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7cc40 0x560150c8d560 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/3398502103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7c250 0x560150c802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3398502103 >> 172.21.15.26:0/3398502103 conn(0x560150c73f40 msgr2=0x560150c6d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3398502103 shutdown_connections 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3398502103 wait complete. 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 -- start start 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 Processor -- start 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7c250 0x560150c8b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7cc40 0x560150cf50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7d6e0 0x560150cf7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560150c94420 con 0x560150c7d6e0 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560150c93d50 con 0x560150c7cc40 2024-04-21T10:30:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560150c94af0 con 0x560150c7c250 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.384+0000 7f7df7fa5740 10 client.0 _ll_register_callbacks cb 0x7f7de400c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7def7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7d6e0 0x560150cf7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7def7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7d6e0 0x560150cf7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52892/0 (socket says 172.21.15.26:52892) 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7def7fe640 1 -- 172.21.15.26:0/3382766940 learned_addr learned my addr 172.21.15.26:0/3382766940 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7deeffd640 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7c250 0x560150c8b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7dee7fc640 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7cc40 0x560150cf50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7def7fe640 1 -- 172.21.15.26:0/3382766940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7c250 msgr2=0x560150c8b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7def7fe640 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7c250 0x560150c8b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7def7fe640 1 -- 172.21.15.26:0/3382766940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7cc40 msgr2=0x560150cf50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7def7fe640 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7cc40 0x560150cf50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7def7fe640 1 -- 172.21.15.26:0/3382766940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560150cf7c20 con 0x560150c7d6e0 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7dee7fc640 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7cc40 0x560150cf50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7def7fe640 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7d6e0 0x560150cf7610 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7de8013260 tx=0x7f7de8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa4794a6f9de9cc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7dd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7dd2ff5640 1 -- 172.21.15.26:0/3382766940 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8024030 con 0x560150c7d6e0 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7dd2ff5640 1 -- 172.21.15.26:0/3382766940 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f7de8004b20 con 0x560150c7d6e0 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3382766940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de4014ab0 con 0x560150c7d6e0 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7dd2ff5640 1 -- 172.21.15.26:0/3382766940 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7de8030050 con 0x560150c7d6e0 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7df7fa5740 10 client.15066 fetch_fsmap learned FSMap version 20 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3382766940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560150cf8fc0 con 0x560150c7d6e0 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.385+0000 7f7dd2ff5640 1 -- 172.21.15.26:0/3382766940 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7de4014ab0 con 0x560150c7d6e0 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.386+0000 7f7dd2ff5640 1 -- 172.21.15.26:0/3382766940 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f7de800f050 con 0x560150c7d6e0 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.386+0000 7f7df7fa5740 10 client.15066 fetch_fsmap finished waiting for FSMap version 20 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.386+0000 7f7df7fa5740 10 client.15066 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.386+0000 7f7df7fa5740 20 client.15066 populate_metadata read hostname 'smithi026' 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.386+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x560150cfdbd0 0x560150cfc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.386+0000 7f7df7fa5740 4 client.15066 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T10:30:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.386+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3382766940 --> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] -- command(tid 0: {"prefix": "status"}) -- 0x560150be43a0 con 0x560150cfdbd0 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.386+0000 7f7deeffd640 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x560150cfdbd0 0x560150cfc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.386+0000 7f7deeffd640 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x560150cfdbd0 0x560150cfc8e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ac2f4e41fad2ba9 server_cookie=e5116a522074b08 in_seq=0 out_seq=0 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.386+0000 7f7dd2ff5640 10 client.15066 ms_handle_connect on v2:172.21.15.190:6836/2091448511 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7dd2ff5640 1 -- 172.21.15.26:0/3382766940 <== mds.2 v2:172.21.15.190:6836/2091448511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560150be43a0 con 0x560150cfdbd0 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7dd2ff5640 10 client.15066 handle_command_reply: tid=0 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df7fa5740 1 client.15066 shutdown 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3382766940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7d6e0 msgr2=0x560150cf7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7d6e0 0x560150cf7610 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7de8013260 tx=0x7f7de8004a40 comp rx=0 tx=0).stop 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3382766940 shutdown_connections 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x560150cfdbd0 0x560150cfc8e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560150c7d6e0 0x560150cf7610 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560150c7cc40 0x560150cf50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df7fa5740 1 --2- 172.21.15.26:0/3382766940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560150c7c250 0x560150c8b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7dd2ff5640 0 client.15066 ms_handle_reset on v2:172.21.15.190:6836/2091448511 2024-04-21T10:30:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3382766940 >> 172.21.15.26:0/3382766940 conn(0x560150c73f40 msgr2=0x560150c80c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3382766940 shutdown_connections 2024-04-21T10:30:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df7fa5740 1 -- 172.21.15.26:0/3382766940 wait complete. 2024-04-21T10:30:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df7fa5740 20 client.15066 trim_cache size 0 max 16384 2024-04-21T10:30:25.393 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:30:25.393 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:30:25.393 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-04-21T10:30:25.393 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24478, 2024-04-21T10:30:25.393 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-21T10:30:25.393 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-21T10:30:25.393 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-21T10:30:25.393 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 80.584438555999995, 2024-04-21T10:30:25.393 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 20, 2024-04-21T10:30:25.394 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 89, 2024-04-21T10:30:25.394 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 87, 2024-04-21T10:30:25.394 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 102.902113203 2024-04-21T10:30:25.394 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:30:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df52d2640 1 -- 172.21.15.26:0/3516340506 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7dd80a3c40 msgr2=0x7f7dd80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df52d2640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7dd80a3c40 0x7f7dd80a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7de8005c40 tx=0x7f7de8013a60 comp rx=0 tx=0).stop 2024-04-21T10:30:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df52d2640 1 -- 172.21.15.26:0/3516340506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df0101fb0 msgr2=0x7f7df0198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df52d2640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df0101fb0 0x7f7df0198290 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7de0000c00 tx=0x7f7de0004270 comp rx=0 tx=0).stop 2024-04-21T10:30:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df52d2640 1 -- 172.21.15.26:0/3516340506 shutdown_connections 2024-04-21T10:30:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df52d2640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7dd80a3c40 0x7f7dd80a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df52d2640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df0103b60 0x7f7df019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df52d2640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7df01031a0 0x7f7df01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df52d2640 1 --2- 172.21.15.26:0/3516340506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7df0101fb0 0x7f7df0198290 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.387+0000 7f7df52d2640 1 -- 172.21.15.26:0/3516340506 >> 172.21.15.26:0/3516340506 conn(0x7f7df00fd520 msgr2=0x7f7df01065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.388+0000 7f7df52d2640 1 -- 172.21.15.26:0/3516340506 shutdown_connections 2024-04-21T10:30:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.388+0000 7f7df52d2640 1 -- 172.21.15.26:0/3516340506 wait complete. 2024-04-21T10:30:25.399 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T10:30:25.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.502+0000 7fd184c3a640 1 -- 172.21.15.26:0/2847027532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180103190 msgr2=0x7fd180103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.502+0000 7fd184c3a640 1 --2- 172.21.15.26:0/2847027532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180103190 0x7fd180103610 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd168009900 tx=0x7fd16802bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:25.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.502+0000 7fd184c3a640 1 -- 172.21.15.26:0/2847027532 shutdown_connections 2024-04-21T10:30:25.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.502+0000 7fd184c3a640 1 --2- 172.21.15.26:0/2847027532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd180103b50 0x7fd18010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.502+0000 7fd184c3a640 1 --2- 172.21.15.26:0/2847027532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180103190 0x7fd180103610 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.502+0000 7fd184c3a640 1 --2- 172.21.15.26:0/2847027532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd180101fa0 0x7fd1801023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.502+0000 7fd184c3a640 1 -- 172.21.15.26:0/2847027532 >> 172.21.15.26:0/2847027532 conn(0x7fd1800fd530 msgr2=0x7fd1800ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.502+0000 7fd184c3a640 1 -- 172.21.15.26:0/2847027532 shutdown_connections 2024-04-21T10:30:25.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.502+0000 7fd184c3a640 1 -- 172.21.15.26:0/2847027532 wait complete. 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd184c3a640 1 Processor -- start 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd184c3a640 1 -- start start 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd184c3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180101fa0 0x7fd1801981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd184c3a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd180103190 0x7fd180198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd184c3a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd180103b50 0x7fd18019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd184c3a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd18010db30 con 0x7fd180101fa0 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd184c3a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd18010d9b0 con 0x7fd180103190 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd184c3a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd18010dcb0 con 0x7fd180103b50 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd17e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180101fa0 0x7fd1801981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd17e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180101fa0 0x7fd1801981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52912/0 (socket says 172.21.15.26:52912) 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd17e59c640 1 -- 172.21.15.26:0/747698627 learned_addr learned my addr 172.21.15.26:0/747698627 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd177fff640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd180103190 0x7fd180198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd17ed9d640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd180103b50 0x7fd18019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd17e59c640 1 -- 172.21.15.26:0/747698627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd180103b50 msgr2=0x7fd18019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd17e59c640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd180103b50 0x7fd18019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd17e59c640 1 -- 172.21.15.26:0/747698627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd180103190 msgr2=0x7fd180198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd17e59c640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd180103190 0x7fd180198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.503+0000 7fd17e59c640 1 -- 172.21.15.26:0/747698627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18019fdf0 con 0x7fd180101fa0 2024-04-21T10:30:25.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.504+0000 7fd177fff640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd180103190 0x7fd180198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:25.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.504+0000 7fd17e59c640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180101fa0 0x7fd1801981d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd17000d950 tx=0x7fd17000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.504+0000 7fd1777fe640 1 -- 172.21.15.26:0/747698627 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170017020 con 0x7fd180101fa0 2024-04-21T10:30:25.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.504+0000 7fd1777fe640 1 -- 172.21.15.26:0/747698627 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd170009c30 con 0x7fd180101fa0 2024-04-21T10:30:25.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.504+0000 7fd184c3a640 1 -- 172.21.15.26:0/747698627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1801a00e0 con 0x7fd180101fa0 2024-04-21T10:30:25.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.504+0000 7fd1777fe640 1 -- 172.21.15.26:0/747698627 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17000f6b0 con 0x7fd180101fa0 2024-04-21T10:30:25.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.504+0000 7fd184c3a640 1 -- 172.21.15.26:0/747698627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1801a0650 con 0x7fd180101fa0 2024-04-21T10:30:25.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.506+0000 7fd1777fe640 1 -- 172.21.15.26:0/747698627 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd17001a020 con 0x7fd180101fa0 2024-04-21T10:30:25.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.506+0000 7fd187822740 1 Processor -- start 2024-04-21T10:30:25.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd187822740 1 -- start start 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd1777fe640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd16c0a3cc0 0x7fd16c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd187822740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5d5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd187822740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5d60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd187822740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d66e0 0x556d6b5dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd187822740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556d6b535b10 con 0x556d6b5d5250 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd187822740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556d6b4d3240 con 0x556d6b5d5c40 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd187822740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556d6b531ad0 con 0x556d6b5d66e0 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd17e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5d5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd17ed9d640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd16c0a3cc0 0x7fd16c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd17e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5d5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52926/0 (socket says 172.21.15.26:52926) 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd17e59c640 1 -- 172.21.15.26:0/4098000371 learned_addr learned my addr 172.21.15.26:0/4098000371 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd1777fe640 1 -- 172.21.15.26:0/747698627 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fd1700d0270 con 0x7fd180101fa0 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd17ed9d640 1 --2- 172.21.15.26:0/4098000371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d66e0 0x556d6b5dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd177fff640 1 --2- 172.21.15.26:0/4098000371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5d60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd17e59c640 1 -- 172.21.15.26:0/4098000371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d66e0 msgr2=0x556d6b5dd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd17e59c640 1 --2- 172.21.15.26:0/4098000371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d66e0 0x556d6b5dd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd17e59c640 1 -- 172.21.15.26:0/4098000371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 msgr2=0x556d6b5d60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd17e59c640 1 --2- 172.21.15.26:0/4098000371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5d60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd17e59c640 1 -- 172.21.15.26:0/4098000371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d6b2482d0 con 0x556d6b5d5250 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd177fff640 1 --2- 172.21.15.26:0/4098000371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5d60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd17ed9d640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd16c0a3cc0 0x7fd16c0a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd178005c40 tx=0x7fd178012a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.507+0000 7fd17e59c640 1 --2- 172.21.15.26:0/4098000371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5d5670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd170009610 tx=0x7fd170010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3efa3e184257fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.508+0000 7fd174ff9640 1 -- 172.21.15.26:0/4098000371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17009cae0 con 0x556d6b5d5250 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.508+0000 7fd174ff9640 1 -- 172.21.15.26:0/4098000371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd170009de0 con 0x556d6b5d5250 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.508+0000 7fd174ff9640 1 -- 172.21.15.26:0/4098000371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170019e30 con 0x556d6b5d5250 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.508+0000 7fd187822740 1 -- 172.21.15.26:0/4098000371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 msgr2=0x556d6b5d5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.508+0000 7fd187822740 1 --2- 172.21.15.26:0/4098000371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5d5670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd170009610 tx=0x7fd170010d40 comp rx=0 tx=0).stop 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.508+0000 7fd187822740 1 -- 172.21.15.26:0/4098000371 shutdown_connections 2024-04-21T10:30:25.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.508+0000 7fd187822740 1 --2- 172.21.15.26:0/4098000371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d66e0 0x556d6b5dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.508+0000 7fd187822740 1 --2- 172.21.15.26:0/4098000371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5d60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.508+0000 7fd187822740 1 --2- 172.21.15.26:0/4098000371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5d5670 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.508+0000 7fd187822740 1 -- 172.21.15.26:0/4098000371 >> 172.21.15.26:0/4098000371 conn(0x556d6b5ccf40 msgr2=0x556d6b5cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd187822740 1 -- 172.21.15.26:0/4098000371 shutdown_connections 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd187822740 1 -- 172.21.15.26:0/4098000371 wait complete. 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd187822740 1 -- start start 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd187822740 1 Processor -- start 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd187822740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5df2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd187822740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5df830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd187822740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d66e0 0x556d6b5e6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd187822740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556d6b5dd8c0 con 0x556d6b5d5250 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd187822740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556d6b4be190 con 0x556d6b5d5c40 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd187822740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556d6b5dda40 con 0x556d6b5d66e0 2024-04-21T10:30:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd187822740 10 client.0 _ll_register_callbacks cb 0x556d6b4b98b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd17e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5df2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd17e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5df2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52932/0 (socket says 172.21.15.26:52932) 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd17e59c640 1 -- 172.21.15.26:0/1374342556 learned_addr learned my addr 172.21.15.26:0/1374342556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd177fff640 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5df830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.509+0000 7fd17ed9d640 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d66e0 0x556d6b5e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd17e59c640 1 -- 172.21.15.26:0/1374342556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d66e0 msgr2=0x556d6b5e6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd17e59c640 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d66e0 0x556d6b5e6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd17e59c640 1 -- 172.21.15.26:0/1374342556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 msgr2=0x556d6b5df830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd17e59c640 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5df830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd17e59c640 1 -- 172.21.15.26:0/1374342556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d6b3126b0 con 0x556d6b5d5250 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd17ed9d640 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d66e0 0x556d6b5e6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd17e59c640 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5df2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd17009c370 tx=0x7fd170066040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321d6325863b900b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:30:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd165ffb640 1 -- 172.21.15.26:0/1374342556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1700a0790 con 0x556d6b5d5250 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd165ffb640 1 -- 172.21.15.26:0/1374342556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fd170098a50 con 0x556d6b5d5250 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd187822740 1 -- 172.21.15.26:0/1374342556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd170009de0 con 0x556d6b5d5250 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd165ffb640 1 -- 172.21.15.26:0/1374342556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd17009cae0 con 0x556d6b5d5250 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd187822740 10 client.15078 fetch_fsmap learned FSMap version 20 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd187822740 1 -- 172.21.15.26:0/1374342556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556d6b5dd8c0 con 0x556d6b5d5250 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd165ffb640 1 -- 172.21.15.26:0/1374342556 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd170009de0 con 0x556d6b5d5250 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd165ffb640 1 -- 172.21.15.26:0/1374342556 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fd170024060 con 0x556d6b5d5250 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd187822740 10 client.15078 fetch_fsmap finished waiting for FSMap version 20 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd187822740 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd187822740 20 client.15078 populate_metadata read hostname 'smithi026' 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd187822740 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x556d6b5ebc20 0x556d6b5ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd187822740 4 client.15078 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.510+0000 7fd187822740 1 -- 172.21.15.26:0/1374342556 --> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d6b543530 con 0x556d6b5ebc20 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.511+0000 7fd177fff640 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x556d6b5ebc20 0x556d6b5ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.511+0000 7fd165ffb640 10 client.15078 ms_handle_connect on v2:172.21.15.169:6838/2428626465 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.511+0000 7fd177fff640 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x556d6b5ebc20 0x556d6b5ed090 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd0464b862d255e8 server_cookie=61f43a60ca79ee1d in_seq=0 out_seq=0 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.512+0000 7fd165ffb640 1 -- 172.21.15.26:0/1374342556 <== mds.1 v2:172.21.15.169:6838/2428626465 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d6b543530 con 0x556d6b5ebc20 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.512+0000 7fd165ffb640 10 client.15078 handle_command_reply: tid=0 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.512+0000 7fd187822740 1 client.15078 shutdown 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.512+0000 7fd187822740 1 -- 172.21.15.26:0/1374342556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 msgr2=0x556d6b5df2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.512+0000 7fd187822740 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5df2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd17009c370 tx=0x7fd170066040 comp rx=0 tx=0).stop 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.513+0000 7fd187822740 1 -- 172.21.15.26:0/1374342556 shutdown_connections 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.513+0000 7fd187822740 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x556d6b5ebc20 0x556d6b5ed090 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.513+0000 7fd187822740 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d66e0 0x556d6b5e6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.513+0000 7fd187822740 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5df830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.513+0000 7fd165ffb640 0 client.15078 ms_handle_reset on v2:172.21.15.169:6838/2428626465 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.513+0000 7fd187822740 1 --2- 172.21.15.26:0/1374342556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5df2f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.513+0000 7fd187822740 1 -- 172.21.15.26:0/1374342556 >> 172.21.15.26:0/1374342556 conn(0x556d6b5ccf40 msgr2=0x556d6b5caa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.513+0000 7fd187822740 1 -- 172.21.15.26:0/1374342556 shutdown_connections 2024-04-21T10:30:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.513+0000 7fd187822740 1 -- 172.21.15.26:0/1374342556 wait complete. 2024-04-21T10:30:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.513+0000 7fd187822740 20 client.15078 trim_cache size 0 max 16384 2024-04-21T10:30:25.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd187822740 1 Processor -- start 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd187822740 1 -- start start 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd187822740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d5250 0x556d6b5ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd187822740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5de320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd187822740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d66e0 0x556d6b5e6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd187822740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556d6b5eb3d0 con 0x556d6b5d66e0 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd187822740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556d6b5ebaa0 con 0x556d6b5d5c40 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd187822740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556d6b5ec170 con 0x556d6b5d5250 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd17ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d66e0 0x556d6b5e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd17ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d66e0 0x556d6b5e6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52940/0 (socket says 172.21.15.26:52940) 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd17ed9d640 1 -- 172.21.15.26:0/331277891 learned_addr learned my addr 172.21.15.26:0/331277891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd17e59c640 1 --2- 172.21.15.26:0/331277891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d5250 0x556d6b5ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd177fff640 1 --2- 172.21.15.26:0/331277891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5de320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd17ed9d640 1 -- 172.21.15.26:0/331277891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d5250 msgr2=0x556d6b5ddde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd17ed9d640 1 --2- 172.21.15.26:0/331277891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d5250 0x556d6b5ddde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd17ed9d640 1 -- 172.21.15.26:0/331277891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 msgr2=0x556d6b5de320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd17ed9d640 1 --2- 172.21.15.26:0/331277891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5de320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.531+0000 7fd17ed9d640 1 -- 172.21.15.26:0/331277891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d6b5dcf70 con 0x556d6b5d66e0 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd17ed9d640 1 --2- 172.21.15.26:0/331277891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d66e0 0x556d6b5e6830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd178003cf0 tx=0x7fd17801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c693281cb4ada9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd167fff640 1 -- 172.21.15.26:0/331277891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17800ee20 con 0x556d6b5d66e0 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd167fff640 1 -- 172.21.15.26:0/331277891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd178024070 con 0x556d6b5d66e0 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd167fff640 1 -- 172.21.15.26:0/331277891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178020970 con 0x556d6b5d66e0 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd187822740 1 -- 172.21.15.26:0/331277891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d66e0 msgr2=0x556d6b5e6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd187822740 1 --2- 172.21.15.26:0/331277891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d66e0 0x556d6b5e6830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd178003cf0 tx=0x7fd17801e000 comp rx=0 tx=0).stop 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd187822740 1 -- 172.21.15.26:0/331277891 shutdown_connections 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd187822740 1 --2- 172.21.15.26:0/331277891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d66e0 0x556d6b5e6830 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd187822740 1 --2- 172.21.15.26:0/331277891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d5c40 0x556d6b5de320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd187822740 1 --2- 172.21.15.26:0/331277891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d5250 0x556d6b5ddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd187822740 1 -- 172.21.15.26:0/331277891 >> 172.21.15.26:0/331277891 conn(0x556d6b5ccf40 msgr2=0x556d6b5c6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd187822740 1 -- 172.21.15.26:0/331277891 shutdown_connections 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.532+0000 7fd187822740 1 -- 172.21.15.26:0/331277891 wait complete. 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd187822740 1 -- start start 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd187822740 1 Processor -- start 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd187822740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd187822740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d5c40 0x556d6b5e4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd187822740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d66e0 0x556d6b5e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd187822740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556d6b5ecf10 con 0x556d6b5d5250 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd187822740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556d6b5ec840 con 0x556d6b5d66e0 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd187822740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556d6b5ed5e0 con 0x556d6b5d5c40 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd187822740 10 client.0 _ll_register_callbacks cb 0x7fd170096f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd17e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd17e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5e2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52950/0 (socket says 172.21.15.26:52950) 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd17e59c640 1 -- 172.21.15.26:0/1642176889 learned_addr learned my addr 172.21.15.26:0/1642176889 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd177fff640 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d5c40 0x556d6b5e4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd17ed9d640 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d66e0 0x556d6b5e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd17e59c640 1 -- 172.21.15.26:0/1642176889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d5c40 msgr2=0x556d6b5e4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd17e59c640 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d5c40 0x556d6b5e4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd17e59c640 1 -- 172.21.15.26:0/1642176889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d66e0 msgr2=0x556d6b5e52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd17e59c640 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d66e0 0x556d6b5e52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd17e59c640 1 -- 172.21.15.26:0/1642176889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d6b5e5990 con 0x556d6b5d5250 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd17e59c640 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5e2810 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd17009beb0 tx=0x7fd17009bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1445bcd58bcb335 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.533+0000 7fd165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd165ffb640 1 -- 172.21.15.26:0/1642176889 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170098680 con 0x556d6b5d5250 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd165ffb640 1 -- 172.21.15.26:0/1642176889 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fd1700b5d60 con 0x556d6b5d5250 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd187822740 1 -- 172.21.15.26:0/1642176889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd178024070 con 0x556d6b5d5250 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd165ffb640 1 -- 172.21.15.26:0/1642176889 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd170009de0 con 0x556d6b5d5250 2024-04-21T10:30:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd187822740 10 client.15084 fetch_fsmap learned FSMap version 20 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd187822740 1 -- 172.21.15.26:0/1642176889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556d6b5e5990 con 0x556d6b5d5250 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd165ffb640 1 -- 172.21.15.26:0/1642176889 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd178024070 con 0x556d6b5d5250 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd165ffb640 1 -- 172.21.15.26:0/1642176889 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fd17009b830 con 0x556d6b5d5250 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd187822740 10 client.15084 fetch_fsmap finished waiting for FSMap version 20 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd187822740 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd187822740 20 client.15084 populate_metadata read hostname 'smithi026' 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd187822740 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x556d6b656490 0x556d6b6551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd187822740 4 client.15084 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd187822740 1 -- 172.21.15.26:0/1642176889 --> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] -- command(tid 0: {"prefix": "status"}) -- 0x556d6b53d3a0 con 0x556d6b656490 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.534+0000 7fd177fff640 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x556d6b656490 0x556d6b6551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd165ffb640 10 client.15084 ms_handle_connect on v2:172.21.15.169:6838/2428626465 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd177fff640 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x556d6b656490 0x556d6b6551a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d054d907a3bb977 server_cookie=4608bd253751ea21 in_seq=0 out_seq=0 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd165ffb640 1 -- 172.21.15.26:0/1642176889 <== mds.1 v2:172.21.15.169:6838/2428626465 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556d6b53d3a0 con 0x556d6b656490 2024-04-21T10:30:25.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd165ffb640 10 client.15084 handle_command_reply: tid=0 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd187822740 1 client.15084 shutdown 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd187822740 1 -- 172.21.15.26:0/1642176889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 msgr2=0x556d6b5e2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd187822740 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5e2810 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd17009beb0 tx=0x7fd17009bd40 comp rx=0 tx=0).stop 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd187822740 1 -- 172.21.15.26:0/1642176889 shutdown_connections 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd187822740 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x556d6b656490 0x556d6b6551a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd187822740 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d6b5d66e0 0x556d6b5e52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd187822740 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556d6b5d5c40 0x556d6b5e4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd165ffb640 0 client.15084 ms_handle_reset on v2:172.21.15.169:6838/2428626465 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd187822740 1 --2- 172.21.15.26:0/1642176889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556d6b5d5250 0x556d6b5e2810 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd187822740 1 -- 172.21.15.26:0/1642176889 >> 172.21.15.26:0/1642176889 conn(0x556d6b5ccf40 msgr2=0x556d6b5d92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd187822740 1 -- 172.21.15.26:0/1642176889 shutdown_connections 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd187822740 1 -- 172.21.15.26:0/1642176889 wait complete. 2024-04-21T10:30:25.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.535+0000 7fd187822740 20 client.15084 trim_cache size 0 max 16384 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24488, 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 81.740022733999993, 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 20, 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 89, 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 87, 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 89.210061103000001 2024-04-21T10:30:25.542 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:30:25.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.536+0000 7fd184c3a640 1 -- 172.21.15.26:0/747698627 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd16c0a3cc0 msgr2=0x7fd16c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.536+0000 7fd184c3a640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd16c0a3cc0 0x7fd16c0a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd178005c40 tx=0x7fd178012a90 comp rx=0 tx=0).stop 2024-04-21T10:30:25.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.536+0000 7fd184c3a640 1 -- 172.21.15.26:0/747698627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180101fa0 msgr2=0x7fd1801981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.536+0000 7fd184c3a640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180101fa0 0x7fd1801981d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd17000d950 tx=0x7fd17000de10 comp rx=0 tx=0).stop 2024-04-21T10:30:25.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.536+0000 7fd184c3a640 1 -- 172.21.15.26:0/747698627 shutdown_connections 2024-04-21T10:30:25.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.536+0000 7fd184c3a640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd16c0a3cc0 0x7fd16c0a6180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.536+0000 7fd184c3a640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd180103b50 0x7fd18019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.536+0000 7fd184c3a640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd180103190 0x7fd180198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.536+0000 7fd184c3a640 1 --2- 172.21.15.26:0/747698627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180101fa0 0x7fd1801981d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.536+0000 7fd184c3a640 1 -- 172.21.15.26:0/747698627 >> 172.21.15.26:0/747698627 conn(0x7fd1800fd530 msgr2=0x7fd180106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.536+0000 7fd184c3a640 1 -- 172.21.15.26:0/747698627 shutdown_connections 2024-04-21T10:30:25.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.536+0000 7fd184c3a640 1 -- 172.21.15.26:0/747698627 wait complete. 2024-04-21T10:30:25.545 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:25 smithi026 ceph-mon[27965]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:25.545 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:25 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1112912556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:25.545 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:25 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3013686111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:25.546 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:25 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1602519648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:25.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T10:30:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.649+0000 7fbb7152b640 1 -- 172.21.15.26:0/4240395084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c100880 msgr2=0x7fbb6c100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.649+0000 7fbb7152b640 1 --2- 172.21.15.26:0/4240395084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c100880 0x7fbb6c100ca0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009900 tx=0x7fbb5c02bcd0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.649+0000 7fbb7152b640 1 -- 172.21.15.26:0/4240395084 shutdown_connections 2024-04-21T10:30:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.649+0000 7fbb7152b640 1 --2- 172.21.15.26:0/4240395084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c101b80 0x7fbb6c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.649+0000 7fbb7152b640 1 --2- 172.21.15.26:0/4240395084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb6c1011e0 0x7fbb6c101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.649+0000 7fbb7152b640 1 --2- 172.21.15.26:0/4240395084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c100880 0x7fbb6c100ca0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.649+0000 7fbb7152b640 1 -- 172.21.15.26:0/4240395084 >> 172.21.15.26:0/4240395084 conn(0x7fbb6c0fc400 msgr2=0x7fbb6c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.649+0000 7fbb7152b640 1 -- 172.21.15.26:0/4240395084 shutdown_connections 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.649+0000 7fbb7152b640 1 -- 172.21.15.26:0/4240395084 wait complete. 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.650+0000 7fbb7152b640 1 Processor -- start 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.650+0000 7fbb7152b640 1 -- start start 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.650+0000 7fbb7152b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb6c100880 0x7fbb6c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.650+0000 7fbb7152b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c1011e0 0x7fbb6c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.650+0000 7fbb7152b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c101b80 0x7fbb6c1a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.650+0000 7fbb7152b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c10fca0 con 0x7fbb6c1011e0 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.650+0000 7fbb7152b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c10fb20 con 0x7fbb6c101b80 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.650+0000 7fbb7152b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c10fe20 con 0x7fbb6c100880 2024-04-21T10:30:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.650+0000 7fbb6a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c1011e0 0x7fbb6c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.650+0000 7fbb6a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c1011e0 0x7fbb6c19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52956/0 (socket says 172.21.15.26:52956) 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.650+0000 7fbb6a7fc640 1 -- 172.21.15.26:0/887424047 learned_addr learned my addr 172.21.15.26:0/887424047 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb6b7fe640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c101b80 0x7fbb6c1a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb6affd640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb6c100880 0x7fbb6c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb6a7fc640 1 -- 172.21.15.26:0/887424047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb6c100880 msgr2=0x7fbb6c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb6c100880 0x7fbb6c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb6a7fc640 1 -- 172.21.15.26:0/887424047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c101b80 msgr2=0x7fbb6c1a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c101b80 0x7fbb6c1a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb6a7fc640 1 -- 172.21.15.26:0/887424047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c1a2050 con 0x7fbb6c1011e0 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c1011e0 0x7fbb6c19a970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000d950 tx=0x7fbb6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb53fff640 1 -- 172.21.15.26:0/887424047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60017030 con 0x7fbb6c1011e0 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb53fff640 1 -- 172.21.15.26:0/887424047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb60004020 con 0x7fbb6c1011e0 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb7152b640 1 -- 172.21.15.26:0/887424047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6c1a2340 con 0x7fbb6c1011e0 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb53fff640 1 -- 172.21.15.26:0/887424047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb600047a0 con 0x7fbb6c1011e0 2024-04-21T10:30:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.651+0000 7fbb7152b640 1 -- 172.21.15.26:0/887424047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6c1a2890 con 0x7fbb6c1011e0 2024-04-21T10:30:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb53fff640 1 -- 172.21.15.26:0/887424047 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbb60009a30 con 0x7fbb6c1011e0 2024-04-21T10:30:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb741fc740 1 Processor -- start 2024-04-21T10:30:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb741fc740 1 -- start start 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb741fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d4895260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb741fc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d4895cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb741fc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 0x5573d489ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb741fc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5573d47f9b10 con 0x5573d48962d0 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb741fc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5573d4797240 con 0x5573d4895830 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb741fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5573d47f5ad0 con 0x5573d4894e40 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb6a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 0x5573d489ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb53fff640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbb540a3c40 0x7fbb540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb6a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 0x5573d489ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52968/0 (socket says 172.21.15.26:52968) 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb6a7fc640 1 -- 172.21.15.26:0/191091764 learned_addr learned my addr 172.21.15.26:0/191091764 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb6b7fe640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbb540a3c40 0x7fbb540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb6b7fe640 1 --2- 172.21.15.26:0/191091764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d4895260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb6affd640 1 --2- 172.21.15.26:0/191091764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d4895cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.654+0000 7fbb53fff640 1 -- 172.21.15.26:0/887424047 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fbb600d0150 con 0x7fbb6c1011e0 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.655+0000 7fbb6a7fc640 1 -- 172.21.15.26:0/191091764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 msgr2=0x5573d4895260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.655+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/191091764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d4895260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.655+0000 7fbb6a7fc640 1 -- 172.21.15.26:0/191091764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 msgr2=0x5573d4895cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.655+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/191091764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d4895cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.655+0000 7fbb6a7fc640 1 -- 172.21.15.26:0/191091764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573d450c2d0 con 0x5573d48962d0 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.655+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/191091764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 0x5573d489ceb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb600cda90 tx=0x7fbb60098620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af39d3af3e30e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.655+0000 7fbb6b7fe640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbb540a3c40 0x7fbb540a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5573d4890200 tx=0x7fbb64004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.655+0000 7fbb517fa640 1 -- 172.21.15.26:0/191091764 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6009bc20 con 0x5573d48962d0 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.655+0000 7fbb517fa640 1 -- 172.21.15.26:0/191091764 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb600a6040 con 0x5573d48962d0 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.655+0000 7fbb517fa640 1 -- 172.21.15.26:0/191091764 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb600cdd40 con 0x5573d48962d0 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 -- 172.21.15.26:0/191091764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 msgr2=0x5573d489ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 --2- 172.21.15.26:0/191091764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 0x5573d489ceb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb600cda90 tx=0x7fbb60098620 comp rx=0 tx=0).stop 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 -- 172.21.15.26:0/191091764 shutdown_connections 2024-04-21T10:30:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 --2- 172.21.15.26:0/191091764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 0x5573d489ceb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 --2- 172.21.15.26:0/191091764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d4895cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 --2- 172.21.15.26:0/191091764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d4895260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 -- 172.21.15.26:0/191091764 >> 172.21.15.26:0/191091764 conn(0x5573d4890f40 msgr2=0x5573d488f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 -- 172.21.15.26:0/191091764 shutdown_connections 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 -- 172.21.15.26:0/191091764 wait complete. 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 -- start start 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 Processor -- start 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4894e40 0x5573d489ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d489f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d48962d0 0x5573d48a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5573d489d450 con 0x5573d4894e40 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5573d4782190 con 0x5573d4895830 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5573d489d5d0 con 0x5573d48962d0 2024-04-21T10:30:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb741fc740 10 client.0 _ll_register_callbacks cb 0x5573d477d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb6affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4894e40 0x5573d489ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb6affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4894e40 0x5573d489ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52972/0 (socket says 172.21.15.26:52972) 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb6affd640 1 -- 172.21.15.26:0/592120548 learned_addr learned my addr 172.21.15.26:0/592120548 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d489f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.656+0000 7fbb6b7fe640 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d48962d0 0x5573d48a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb6affd640 1 -- 172.21.15.26:0/592120548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d48962d0 msgr2=0x5573d48a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb6affd640 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d48962d0 0x5573d48a63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb6affd640 1 -- 172.21.15.26:0/592120548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 msgr2=0x5573d489f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb6affd640 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d489f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb6affd640 1 -- 172.21.15.26:0/592120548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573d45d66b0 con 0x5573d4894e40 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb6affd640 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4894e40 0x5573d489ee50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c005ab0 tx=0x7fbb5c02dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e18134004d1be35 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb4eff5640 1 -- 172.21.15.26:0/592120548 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c043030 con 0x5573d4894e40 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb4eff5640 1 -- 172.21.15.26:0/592120548 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fbb5c005250 con 0x5573d4894e40 2024-04-21T10:30:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb741fc740 1 -- 172.21.15.26:0/592120548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb600a6040 con 0x5573d4894e40 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb4eff5640 1 -- 172.21.15.26:0/592120548 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbb5c005d50 con 0x5573d4894e40 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb741fc740 10 client.15096 fetch_fsmap learned FSMap version 20 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb741fc740 1 -- 172.21.15.26:0/592120548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5573d48a97b0 con 0x5573d4894e40 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb4eff5640 1 -- 172.21.15.26:0/592120548 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb600a6040 con 0x5573d4894e40 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb4eff5640 1 -- 172.21.15.26:0/592120548 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fbb5c0036d0 con 0x5573d4894e40 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb741fc740 10 client.15096 fetch_fsmap finished waiting for FSMap version 20 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.657+0000 7fbb741fc740 10 client.15096 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.658+0000 7fbb741fc740 20 client.15096 populate_metadata read hostname 'smithi026' 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.658+0000 7fbb741fc740 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x5573d48abe60 0x5573d48ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.658+0000 7fbb741fc740 4 client.15096 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.658+0000 7fbb741fc740 1 -- 172.21.15.26:0/592120548 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573d4807530 con 0x5573d48abe60 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.658+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x5573d48abe60 0x5573d48ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.658+0000 7fbb4eff5640 10 client.15096 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:30:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.658+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x5573d48abe60 0x5573d48ad2d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c41a4f4da632ee8b server_cookie=e75dbca9d8a704f4 in_seq=0 out_seq=0 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.659+0000 7fbb4eff5640 1 -- 172.21.15.26:0/592120548 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573d4807530 con 0x5573d48abe60 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.659+0000 7fbb4eff5640 10 client.15096 handle_command_reply: tid=0 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.659+0000 7fbb741fc740 1 client.15096 shutdown 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.660+0000 7fbb741fc740 1 -- 172.21.15.26:0/592120548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4894e40 msgr2=0x5573d489ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.660+0000 7fbb741fc740 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4894e40 0x5573d489ee50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c005ab0 tx=0x7fbb5c02dba0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.660+0000 7fbb741fc740 1 -- 172.21.15.26:0/592120548 shutdown_connections 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.660+0000 7fbb741fc740 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x5573d48abe60 0x5573d48ad2d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.660+0000 7fbb741fc740 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d48962d0 0x5573d48a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.660+0000 7fbb741fc740 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d489f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.660+0000 7fbb741fc740 1 --2- 172.21.15.26:0/592120548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4894e40 0x5573d489ee50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.660+0000 7fbb4eff5640 0 client.15096 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.660+0000 7fbb741fc740 1 -- 172.21.15.26:0/592120548 >> 172.21.15.26:0/592120548 conn(0x5573d4890f40 msgr2=0x5573d488ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.660+0000 7fbb741fc740 1 -- 172.21.15.26:0/592120548 shutdown_connections 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.660+0000 7fbb741fc740 1 -- 172.21.15.26:0/592120548 wait complete. 2024-04-21T10:30:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.660+0000 7fbb741fc740 20 client.15096 trim_cache size 0 max 16384 2024-04-21T10:30:25.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.678+0000 7fbb741fc740 1 Processor -- start 2024-04-21T10:30:25.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.678+0000 7fbb741fc740 1 -- start start 2024-04-21T10:30:25.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.678+0000 7fbb741fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d4899840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.678+0000 7fbb741fc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4895830 0x5573d4899d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.678+0000 7fbb741fc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d48962d0 0x5573d48a6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.678+0000 7fbb741fc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5573d48ab610 con 0x5573d4895830 2024-04-21T10:30:25.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.678+0000 7fbb741fc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5573d48abce0 con 0x5573d48962d0 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.678+0000 7fbb741fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5573d48ac3b0 con 0x5573d4894e40 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.678+0000 7fbb6a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4895830 0x5573d4899d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.678+0000 7fbb6a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4895830 0x5573d4899d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52980/0 (socket says 172.21.15.26:52980) 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.678+0000 7fbb6a7fc640 1 -- 172.21.15.26:0/2737497457 learned_addr learned my addr 172.21.15.26:0/2737497457 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.679+0000 7fbb6affd640 1 --2- 172.21.15.26:0/2737497457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d4899840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.679+0000 7fbb6a7fc640 1 -- 172.21.15.26:0/2737497457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 msgr2=0x5573d4899840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.679+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/2737497457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d4899840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.679+0000 7fbb6b7fe640 1 --2- 172.21.15.26:0/2737497457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d48962d0 0x5573d48a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.679+0000 7fbb6a7fc640 1 -- 172.21.15.26:0/2737497457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d48962d0 msgr2=0x5573d48a6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.679+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/2737497457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d48962d0 0x5573d48a6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.679+0000 7fbb6a7fc640 1 -- 172.21.15.26:0/2737497457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573d4890230 con 0x5573d4895830 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.679+0000 7fbb6affd640 1 --2- 172.21.15.26:0/2737497457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d4899840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.679+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/2737497457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4895830 0x5573d4899d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbb60005fd0 tx=0x7fbb60096d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f97b34022bf38c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.679+0000 7fbb50ff9640 1 -- 172.21.15.26:0/2737497457 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb600b5020 con 0x5573d4895830 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.679+0000 7fbb50ff9640 1 -- 172.21.15.26:0/2737497457 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb600a6040 con 0x5573d4895830 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.679+0000 7fbb50ff9640 1 -- 172.21.15.26:0/2737497457 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb600041d0 con 0x5573d4895830 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 -- 172.21.15.26:0/2737497457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4895830 msgr2=0x5573d4899d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 --2- 172.21.15.26:0/2737497457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4895830 0x5573d4899d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbb60005fd0 tx=0x7fbb60096d20 comp rx=0 tx=0).stop 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 -- 172.21.15.26:0/2737497457 shutdown_connections 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 --2- 172.21.15.26:0/2737497457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d48962d0 0x5573d48a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 --2- 172.21.15.26:0/2737497457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d4895830 0x5573d4899d80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 --2- 172.21.15.26:0/2737497457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d4899840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 -- 172.21.15.26:0/2737497457 >> 172.21.15.26:0/2737497457 conn(0x5573d4890f40 msgr2=0x5573d488a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 -- 172.21.15.26:0/2737497457 shutdown_connections 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 -- 172.21.15.26:0/2737497457 wait complete. 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 -- start start 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 Processor -- start 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d48a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d48a4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 0x5573d48a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5573d48ad150 con 0x5573d48962d0 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5573d48aca80 con 0x5573d4895830 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5573d48ad820 con 0x5573d4894e40 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb741fc740 10 client.0 _ll_register_callbacks cb 0x7fbb5c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb6b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 0x5573d48a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb6b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 0x5573d48a4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52994/0 (socket says 172.21.15.26:52994) 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb6b7fe640 1 -- 172.21.15.26:0/3587348432 learned_addr learned my addr 172.21.15.26:0/3587348432 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb6affd640 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d48a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d48a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb6b7fe640 1 -- 172.21.15.26:0/3587348432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 msgr2=0x5573d48a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb6b7fe640 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d48a22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb6b7fe640 1 -- 172.21.15.26:0/3587348432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 msgr2=0x5573d48a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.680+0000 7fbb6b7fe640 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d48a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb6b7fe640 1 -- 172.21.15.26:0/3587348432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573d48a56d0 con 0x5573d48962d0 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb6a7fc640 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d48a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb6b7fe640 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 0x5573d48a4ea0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbb64005fd0 tx=0x7fbb64004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b2473cec003845 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb4eff5640 1 -- 172.21.15.26:0/3587348432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64022c90 con 0x5573d48962d0 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb4eff5640 1 -- 172.21.15.26:0/3587348432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fbb64004b20 con 0x5573d48962d0 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb741fc740 1 -- 172.21.15.26:0/3587348432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb600a6040 con 0x5573d48962d0 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb4eff5640 1 -- 172.21.15.26:0/3587348432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbb64022e30 con 0x5573d48962d0 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb741fc740 10 client.15102 fetch_fsmap learned FSMap version 20 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb741fc740 1 -- 172.21.15.26:0/3587348432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5573d4911ee0 con 0x5573d48962d0 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb4eff5640 1 -- 172.21.15.26:0/3587348432 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb600a6040 con 0x5573d48962d0 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb4eff5640 1 -- 172.21.15.26:0/3587348432 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fbb64029030 con 0x5573d48962d0 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb741fc740 10 client.15102 fetch_fsmap finished waiting for FSMap version 20 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb741fc740 10 client.15102 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T10:30:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb741fc740 20 client.15102 populate_metadata read hostname 'smithi026' 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb741fc740 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x5573d4916b50 0x5573d4912f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb741fc740 4 client.15102 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.681+0000 7fbb741fc740 1 -- 172.21.15.26:0/3587348432 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "status"}) -- 0x5573d48013a0 con 0x5573d4916b50 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb6affd640 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x5573d4916b50 0x5573d4912f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb4eff5640 10 client.15102 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb6affd640 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x5573d4916b50 0x5573d4912f90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0f188b15f04f83 server_cookie=4925d24aff8c7c4f in_seq=0 out_seq=0 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb4eff5640 1 -- 172.21.15.26:0/3587348432 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5573d48013a0 con 0x5573d4916b50 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb4eff5640 10 client.15102 handle_command_reply: tid=0 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb741fc740 1 client.15102 shutdown 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb741fc740 1 -- 172.21.15.26:0/3587348432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 msgr2=0x5573d48a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb741fc740 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 0x5573d48a4ea0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbb64005fd0 tx=0x7fbb64004ab0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb741fc740 1 -- 172.21.15.26:0/3587348432 shutdown_connections 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb741fc740 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x5573d4916b50 0x5573d4912f90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb741fc740 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573d48962d0 0x5573d48a4ea0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb741fc740 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5573d4895830 0x5573d48a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb4eff5640 0 client.15102 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:30:25.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb741fc740 1 --2- 172.21.15.26:0/3587348432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5573d4894e40 0x5573d48a22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.682+0000 7fbb741fc740 1 -- 172.21.15.26:0/3587348432 >> 172.21.15.26:0/3587348432 conn(0x5573d4890f40 msgr2=0x5573d488e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb741fc740 1 -- 172.21.15.26:0/3587348432 shutdown_connections 2024-04-21T10:30:25.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb741fc740 1 -- 172.21.15.26:0/3587348432 wait complete. 2024-04-21T10:30:25.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb741fc740 20 client.15102 trim_cache size 0 max 16384 2024-04-21T10:30:25.689 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:30:25.689 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:30:25.689 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-21T10:30:25.689 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24496, 2024-04-21T10:30:25.690 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-21T10:30:25.690 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-21T10:30:25.690 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-21T10:30:25.690 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 83.770704238999997, 2024-04-21T10:30:25.690 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 20, 2024-04-21T10:30:25.690 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 89, 2024-04-21T10:30:25.690 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 87, 2024-04-21T10:30:25.690 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 97.718859750999997 2024-04-21T10:30:25.690 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:30:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb7152b640 1 -- 172.21.15.26:0/887424047 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbb540a3c40 msgr2=0x7fbb540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb7152b640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbb540a3c40 0x7fbb540a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5573d4890200 tx=0x7fbb64004a40 comp rx=0 tx=0).stop 2024-04-21T10:30:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb7152b640 1 -- 172.21.15.26:0/887424047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c1011e0 msgr2=0x7fbb6c19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb7152b640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c1011e0 0x7fbb6c19a970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000d950 tx=0x7fbb6000de10 comp rx=0 tx=0).stop 2024-04-21T10:30:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb7152b640 1 -- 172.21.15.26:0/887424047 shutdown_connections 2024-04-21T10:30:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb7152b640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbb540a3c40 0x7fbb540a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb7152b640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb6c101b80 0x7fbb6c1a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb7152b640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c1011e0 0x7fbb6c19a970 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb7152b640 1 --2- 172.21.15.26:0/887424047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb6c100880 0x7fbb6c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:25.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb7152b640 1 -- 172.21.15.26:0/887424047 >> 172.21.15.26:0/887424047 conn(0x7fbb6c0fc400 msgr2=0x7fbb6c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:25.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb7152b640 1 -- 172.21.15.26:0/887424047 shutdown_connections 2024-04-21T10:30:25.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:25.683+0000 7fbb7152b640 1 -- 172.21.15.26:0/887424047 wait complete. 2024-04-21T10:30:25.694 INFO:tasks.cephadm:Setup complete, yielding 2024-04-21T10:30:25.694 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-21T10:30:25.704 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-04-21T10:30:25.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-21T10:30:25.705 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:25 smithi169 ceph-mon[31469]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:25.705 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:25 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1112912556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:25.705 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:25 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3013686111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:25.705 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:25 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1602519648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:25.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:25 smithi190 ceph-mon[31423]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:25.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:25 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1112912556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:25.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:25 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3013686111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:25.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:25 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1602519648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:25.951 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:27.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.615+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2698248649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd01087b0 msgr2=0x7fcfd010ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.615+0000 7fcfd6e81700 1 --2- 172.21.15.26:0/2698248649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd01087b0 0x7fcfd010ac20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800b0a0 tx=0x7fcfc800b3b0 comp rx=0 tx=0).stop 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.615+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2698248649 shutdown_connections 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.615+0000 7fcfd6e81700 1 --2- 172.21.15.26:0/2698248649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfd0103550 0x7fcfd0105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.615+0000 7fcfd6e81700 1 --2- 172.21.15.26:0/2698248649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd01087b0 0x7fcfd010ac20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.615+0000 7fcfd6e81700 1 --2- 172.21.15.26:0/2698248649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd0105e80 0x7fcfd0108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.615+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2698248649 >> 172.21.15.26:0/2698248649 conn(0x7fcfd00fcf60 msgr2=0x7fcfd00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.615+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2698248649 shutdown_connections 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.615+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2698248649 wait complete. 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfd6e81700 1 Processor -- start 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfd6e81700 1 -- start start 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfd6e81700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfd0103550 0x7fcfd0102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfd6e81700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd0105e80 0x7fcfd0101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfd6e81700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd01087b0 0x7fcfd0101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfd6e81700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd010e5f0 con 0x7fcfd0105e80 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfd6e81700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd010e470 con 0x7fcfd0103550 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfd6e81700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd010e770 con 0x7fcfd01087b0 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfcffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd0105e80 0x7fcfd0101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfcffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd0105e80 0x7fcfd0101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53008/0 (socket says 172.21.15.26:53008) 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfcffff700 1 -- 172.21.15.26:0/2601641247 learned_addr learned my addr 172.21.15.26:0/2601641247 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfd541e700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd01087b0 0x7fcfd0101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfd4c1d700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfd0103550 0x7fcfd0102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfcffff700 1 -- 172.21.15.26:0/2601641247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd01087b0 msgr2=0x7fcfd0101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfcffff700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd01087b0 0x7fcfd0101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfcffff700 1 -- 172.21.15.26:0/2601641247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfd0103550 msgr2=0x7fcfd0102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfcffff700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfd0103550 0x7fcfd0102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfcffff700 1 -- 172.21.15.26:0/2601641247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd0102250 con 0x7fcfd0105e80 2024-04-21T10:30:27.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.616+0000 7fcfd4c1d700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfd0103550 0x7fcfd0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:27.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.617+0000 7fcfcffff700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd0105e80 0x7fcfd0101500 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00c920 tx=0x7fcfbc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:27.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.617+0000 7fcfcdffb700 1 -- 172.21.15.26:0/2601641247 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc007b00 con 0x7fcfd0105e80 2024-04-21T10:30:27.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.617+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2601641247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd01a2da0 con 0x7fcfd0105e80 2024-04-21T10:30:27.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.617+0000 7fcfcdffb700 1 -- 172.21.15.26:0/2601641247 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfbc004ce0 con 0x7fcfd0105e80 2024-04-21T10:30:27.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.617+0000 7fcfcdffb700 1 -- 172.21.15.26:0/2601641247 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc005650 con 0x7fcfd0105e80 2024-04-21T10:30:27.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.617+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2601641247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd01a32d0 con 0x7fcfd0105e80 2024-04-21T10:30:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.618+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2601641247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd0069730 con 0x7fcfd0105e80 2024-04-21T10:30:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.619+0000 7fcfcdffb700 1 -- 172.21.15.26:0/2601641247 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcfbc004750 con 0x7fcfd0105e80 2024-04-21T10:30:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.620+0000 7fcfcdffb700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcfc00a3cd0 0x7fcfc00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.620+0000 7fcfd4c1d700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcfc00a3cd0 0x7fcfc00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.620+0000 7fcfcdffb700 1 -- 172.21.15.26:0/2601641247 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fcfbc0d0250 con 0x7fcfd0105e80 2024-04-21T10:30:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.620+0000 7fcfd4c1d700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcfc00a3cd0 0x7fcfc00a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4007900 tx=0x7fcfc4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:27.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.624+0000 7fcfcdffb700 1 -- 172.21.15.26:0/2601641247 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcfbc014030 con 0x7fcfd0105e80 2024-04-21T10:30:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:27 smithi026 ceph-mon[27965]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:27.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:27 smithi169 ceph-mon[31469]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:27.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:27 smithi190 ceph-mon[31423]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:27.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.856+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2601641247 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fcfd0062e80 con 0x7fcfc00a3cd0 2024-04-21T10:30:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.857+0000 7fcfcdffb700 1 -- 172.21.15.26:0/2601641247 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fcfd0062e80 con 0x7fcfc00a3cd0 2024-04-21T10:30:27.857 INFO:teuthology.orchestra.run.smithi026.stdout:Backend: cephadm 2024-04-21T10:30:27.857 INFO:teuthology.orchestra.run.smithi026.stdout:Available: Yes 2024-04-21T10:30:27.857 INFO:teuthology.orchestra.run.smithi026.stdout:Paused: No 2024-04-21T10:30:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.860+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2601641247 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcfc00a3cd0 msgr2=0x7fcfc00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.860+0000 7fcfd6e81700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcfc00a3cd0 0x7fcfc00a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4007900 tx=0x7fcfc4008040 comp rx=0 tx=0).stop 2024-04-21T10:30:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.860+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2601641247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd0105e80 msgr2=0x7fcfd0101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.860+0000 7fcfd6e81700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd0105e80 0x7fcfd0101500 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00c920 tx=0x7fcfbc00cce0 comp rx=0 tx=0).stop 2024-04-21T10:30:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.860+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2601641247 shutdown_connections 2024-04-21T10:30:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.860+0000 7fcfd6e81700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcfc00a3cd0 0x7fcfc00a6190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.860+0000 7fcfd6e81700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfd0103550 0x7fcfd0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.860+0000 7fcfd6e81700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd01087b0 0x7fcfd0101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.860+0000 7fcfd6e81700 1 --2- 172.21.15.26:0/2601641247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd0105e80 0x7fcfd0101500 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.860+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2601641247 >> 172.21.15.26:0/2601641247 conn(0x7fcfd00fcf60 msgr2=0x7fcfd0103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.860+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2601641247 shutdown_connections 2024-04-21T10:30:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:27.860+0000 7fcfd6e81700 1 -- 172.21.15.26:0/2601641247 wait complete. 2024-04-21T10:30:28.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:28 smithi026 ceph-mon[27965]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:28.484 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-21T10:30:28.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:28 smithi169 ceph-mon[31469]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:28.737 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:28.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:28 smithi190 ceph-mon[31423]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:29.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:29 smithi026 ceph-mon[27965]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:30:29.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:29 smithi169 ceph-mon[31469]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:30:29.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:29 smithi190 ceph-mon[31423]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:30:30.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.335+0000 7f1166fd9700 1 -- 172.21.15.26:0/1401917121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1160103ca0 msgr2=0x7f1160104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.335+0000 7f1166fd9700 1 --2- 172.21.15.26:0/1401917121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1160103ca0 0x7f1160104100 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f114c0099c0 tx=0x7f114c009cd0 comp rx=0 tx=0).stop 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.335+0000 7f1166fd9700 1 -- 172.21.15.26:0/1401917121 shutdown_connections 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.335+0000 7f1166fd9700 1 --2- 172.21.15.26:0/1401917121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1160102aa0 0x7f1160102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.335+0000 7f1166fd9700 1 --2- 172.21.15.26:0/1401917121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1160104640 0x7f116010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.335+0000 7f1166fd9700 1 --2- 172.21.15.26:0/1401917121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1160103ca0 0x7f1160104100 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.335+0000 7f1166fd9700 1 -- 172.21.15.26:0/1401917121 >> 172.21.15.26:0/1401917121 conn(0x7f11600fe040 msgr2=0x7f1160100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.335+0000 7f1166fd9700 1 -- 172.21.15.26:0/1401917121 shutdown_connections 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.335+0000 7f1166fd9700 1 -- 172.21.15.26:0/1401917121 wait complete. 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.335+0000 7f1166fd9700 1 Processor -- start 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1166fd9700 1 -- start start 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1166fd9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1160102aa0 0x7f1160198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1166fd9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1160103ca0 0x7f11601990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1166fd9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1160104640 0x7f11601a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1166fd9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f116010e540 con 0x7f1160104640 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1166fd9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f116010e3c0 con 0x7f1160103ca0 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1166fd9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f116010e6c0 con 0x7f1160102aa0 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1165576700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1160104640 0x7f11601a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1165576700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1160104640 0x7f11601a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53028/0 (socket says 172.21.15.26:53028) 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1165576700 1 -- 172.21.15.26:0/1223761923 learned_addr learned my addr 172.21.15.26:0/1223761923 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1164d75700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1160102aa0 0x7f1160198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f115ffff700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1160103ca0 0x7f11601990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1165576700 1 -- 172.21.15.26:0/1223761923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1160102aa0 msgr2=0x7f1160198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1165576700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1160102aa0 0x7f1160198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1165576700 1 -- 172.21.15.26:0/1223761923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1160103ca0 msgr2=0x7f11601990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1165576700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1160103ca0 0x7f11601990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1165576700 1 -- 172.21.15.26:0/1223761923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11601a07c0 con 0x7f1160104640 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f115ffff700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1160103ca0 0x7f11601990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1164d75700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1160102aa0 0x7f1160198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:30.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.336+0000 7f1165576700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1160104640 0x7f11601a0120 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1158007f10 tx=0x7f115800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:30.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.337+0000 7f115dffb700 1 -- 172.21.15.26:0/1223761923 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158004bb0 con 0x7f1160104640 2024-04-21T10:30:30.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.337+0000 7f1166fd9700 1 -- 172.21.15.26:0/1223761923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11601a0ab0 con 0x7f1160104640 2024-04-21T10:30:30.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.337+0000 7f115dffb700 1 -- 172.21.15.26:0/1223761923 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1158004d50 con 0x7f1160104640 2024-04-21T10:30:30.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.337+0000 7f115dffb700 1 -- 172.21.15.26:0/1223761923 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158005a30 con 0x7f1160104640 2024-04-21T10:30:30.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.337+0000 7f1166fd9700 1 -- 172.21.15.26:0/1223761923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11601a0ff0 con 0x7f1160104640 2024-04-21T10:30:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.338+0000 7f1166fd9700 1 -- 172.21.15.26:0/1223761923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1160069730 con 0x7f1160104640 2024-04-21T10:30:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.339+0000 7f115dffb700 1 -- 172.21.15.26:0/1223761923 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f115801a020 con 0x7f1160104640 2024-04-21T10:30:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.339+0000 7f115dffb700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f11500a3cd0 0x7f11500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.340+0000 7f115dffb700 1 -- 172.21.15.26:0/1223761923 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f11580cfd50 con 0x7f1160104640 2024-04-21T10:30:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.340+0000 7f1164d75700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f11500a3cd0 0x7f11500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:30.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.340+0000 7f1164d75700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f11500a3cd0 0x7f11500a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1160103b00 tx=0x7f115400a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.344+0000 7f115dffb700 1 -- 172.21.15.26:0/1223761923 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1158010070 con 0x7f1160104640 2024-04-21T10:30:30.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.563+0000 7f1166fd9700 1 -- 172.21.15.26:0/1223761923 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f1160062e80 con 0x7f11500a3cd0 2024-04-21T10:30:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.576+0000 7f115dffb700 1 -- 172.21.15.26:0/1223761923 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f1160062e80 con 0x7f11500a3cd0 2024-04-21T10:30:30.577 INFO:teuthology.orchestra.run.smithi026.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-21T10:30:30.577 INFO:teuthology.orchestra.run.smithi026.stdout:mds.a smithi026 running (97s) 15s ago 96s 16.2M - 19.0.0-3154-ga630765c b45a994450e2 862ab8ba0c5b 2024-04-21T10:30:30.577 INFO:teuthology.orchestra.run.smithi026.stdout:mds.b smithi169 running (94s) 27s ago 94s 19.3M - 19.0.0-3154-ga630765c b45a994450e2 9b7427cf3b11 2024-04-21T10:30:30.577 INFO:teuthology.orchestra.run.smithi026.stdout:mds.c smithi190 running (108s) 27s ago 108s 17.1M - 19.0.0-3154-ga630765c b45a994450e2 945f9b9fd7f4 2024-04-21T10:30:30.577 INFO:teuthology.orchestra.run.smithi026.stdout:mds.d smithi026 running (116s) 15s ago 116s 15.4M - 19.0.0-3154-ga630765c b45a994450e2 3675e1e3c425 2024-04-21T10:30:30.578 INFO:teuthology.orchestra.run.smithi026.stdout:mds.e smithi169 running (113s) 27s ago 113s 18.2M - 19.0.0-3154-ga630765c b45a994450e2 ed196f30de18 2024-04-21T10:30:30.578 INFO:teuthology.orchestra.run.smithi026.stdout:mds.f smithi190 running (111s) 27s ago 110s 20.5M - 19.0.0-3154-ga630765c b45a994450e2 2a1ab1cd00df 2024-04-21T10:30:30.578 INFO:teuthology.orchestra.run.smithi026.stdout:mds.g smithi026 running (100s) 15s ago 99s 13.7M - 19.0.0-3154-ga630765c b45a994450e2 9494afbdcc14 2024-04-21T10:30:30.578 INFO:teuthology.orchestra.run.smithi026.stdout:mds.h smithi169 running (105s) 27s ago 105s 16.6M - 19.0.0-3154-ga630765c b45a994450e2 80036354afb8 2024-04-21T10:30:30.578 INFO:teuthology.orchestra.run.smithi026.stdout:mds.i smithi190 running (102s) 27s ago 102s 25.1M - 19.0.0-3154-ga630765c b45a994450e2 fc172937a855 2024-04-21T10:30:30.578 INFO:teuthology.orchestra.run.smithi026.stdout:mds.j smithi026 running (119s) 15s ago 119s 16.2M - 19.0.0-3154-ga630765c b45a994450e2 d71fb6b067d4 2024-04-21T10:30:30.579 INFO:teuthology.orchestra.run.smithi026.stdout:mds.k smithi169 running (2m) 27s ago 2m 16.4M - 19.0.0-3154-ga630765c b45a994450e2 58f7521f3994 2024-04-21T10:30:30.579 INFO:teuthology.orchestra.run.smithi026.stdout:mds.l smithi190 running (2m) 27s ago 2m 13.9M - 19.0.0-3154-ga630765c b45a994450e2 68236a017c6d 2024-04-21T10:30:30.579 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.x smithi026 *:9283,8765 running (9m) 15s ago 9m 514M - 19.0.0-3154-ga630765c b45a994450e2 ee7c00245b82 2024-04-21T10:30:30.579 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.y smithi169 *:8443,8765 running (7m) 27s ago 7m 452M - 19.0.0-3154-ga630765c b45a994450e2 d677f561df8a 2024-04-21T10:30:30.579 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.z smithi190 *:8443,8765 running (7m) 27s ago 7m 451M - 19.0.0-3154-ga630765c b45a994450e2 189f62033133 2024-04-21T10:30:30.579 INFO:teuthology.orchestra.run.smithi026.stdout:mon.a smithi026 running (9m) 15s ago 9m 52.1M 2048M 19.0.0-3154-ga630765c b45a994450e2 1526264d8981 2024-04-21T10:30:30.580 INFO:teuthology.orchestra.run.smithi026.stdout:mon.b smithi169 running (7m) 27s ago 7m 41.3M 2048M 19.0.0-3154-ga630765c b45a994450e2 4f5a02b35bd3 2024-04-21T10:30:30.580 INFO:teuthology.orchestra.run.smithi026.stdout:mon.c smithi190 running (7m) 27s ago 7m 38.6M 2048M 19.0.0-3154-ga630765c b45a994450e2 5f7e1da45384 2024-04-21T10:30:30.580 INFO:teuthology.orchestra.run.smithi026.stdout:osd.0 smithi026 running (7m) 15s ago 7m 48.2M 4273M 19.0.0-3154-ga630765c b45a994450e2 56c9fc28a68e 2024-04-21T10:30:30.580 INFO:teuthology.orchestra.run.smithi026.stdout:osd.1 smithi169 running (6m) 27s ago 6m 59.1M 4272M 19.0.0-3154-ga630765c b45a994450e2 6b38493a1b26 2024-04-21T10:30:30.580 INFO:teuthology.orchestra.run.smithi026.stdout:osd.2 smithi190 running (6m) 27s ago 6m 48.9M 4272M 19.0.0-3154-ga630765c b45a994450e2 2ab2182e110d 2024-04-21T10:30:30.580 INFO:teuthology.orchestra.run.smithi026.stdout:osd.3 smithi026 running (5m) 15s ago 5m 47.3M 4273M 19.0.0-3154-ga630765c b45a994450e2 dfe0383836b6 2024-04-21T10:30:30.580 INFO:teuthology.orchestra.run.smithi026.stdout:osd.4 smithi169 running (5m) 27s ago 5m 53.4M 4272M 19.0.0-3154-ga630765c b45a994450e2 fe92112709cc 2024-04-21T10:30:30.581 INFO:teuthology.orchestra.run.smithi026.stdout:osd.5 smithi190 running (4m) 27s ago 4m 55.4M 4272M 19.0.0-3154-ga630765c b45a994450e2 a23a31fdfbcc 2024-04-21T10:30:30.581 INFO:teuthology.orchestra.run.smithi026.stdout:osd.6 smithi026 running (4m) 15s ago 4m 47.8M 4273M 19.0.0-3154-ga630765c b45a994450e2 4bedfe059385 2024-04-21T10:30:30.581 INFO:teuthology.orchestra.run.smithi026.stdout:osd.7 smithi169 running (4m) 27s ago 4m 45.1M 4272M 19.0.0-3154-ga630765c b45a994450e2 31b0e85c2636 2024-04-21T10:30:30.581 INFO:teuthology.orchestra.run.smithi026.stdout:osd.8 smithi190 running (3m) 27s ago 3m 46.6M 4272M 19.0.0-3154-ga630765c b45a994450e2 792ef178830c 2024-04-21T10:30:30.581 INFO:teuthology.orchestra.run.smithi026.stdout:osd.9 smithi026 running (3m) 15s ago 3m 45.7M 4273M 19.0.0-3154-ga630765c b45a994450e2 2fb66ec07df0 2024-04-21T10:30:30.581 INFO:teuthology.orchestra.run.smithi026.stdout:osd.10 smithi169 running (2m) 27s ago 2m 49.2M 4272M 19.0.0-3154-ga630765c b45a994450e2 25fc627952eb 2024-04-21T10:30:30.582 INFO:teuthology.orchestra.run.smithi026.stdout:osd.11 smithi190 running (2m) 27s ago 2m 52.7M 4272M 19.0.0-3154-ga630765c b45a994450e2 6429ceabb85c 2024-04-21T10:30:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.578+0000 7f1166fd9700 1 -- 172.21.15.26:0/1223761923 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f11500a3cd0 msgr2=0x7f11500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.578+0000 7f1166fd9700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f11500a3cd0 0x7f11500a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1160103b00 tx=0x7f115400a680 comp rx=0 tx=0).stop 2024-04-21T10:30:30.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.578+0000 7f1166fd9700 1 -- 172.21.15.26:0/1223761923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1160104640 msgr2=0x7f11601a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:30.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.578+0000 7f1166fd9700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1160104640 0x7f11601a0120 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1158007f10 tx=0x7f115800d3b0 comp rx=0 tx=0).stop 2024-04-21T10:30:30.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.578+0000 7f1166fd9700 1 -- 172.21.15.26:0/1223761923 shutdown_connections 2024-04-21T10:30:30.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.578+0000 7f1166fd9700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f11500a3cd0 0x7f11500a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:30.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.578+0000 7f1166fd9700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1160104640 0x7f11601a0120 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:30.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.578+0000 7f1166fd9700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1160102aa0 0x7f1160198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:30.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.578+0000 7f1166fd9700 1 --2- 172.21.15.26:0/1223761923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1160103ca0 0x7f11601990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:30.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.578+0000 7f1166fd9700 1 -- 172.21.15.26:0/1223761923 >> 172.21.15.26:0/1223761923 conn(0x7f11600fe040 msgr2=0x7f1160106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:30.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.579+0000 7f1166fd9700 1 -- 172.21.15.26:0/1223761923 shutdown_connections 2024-04-21T10:30:30.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:30.579+0000 7f1166fd9700 1 -- 172.21.15.26:0/1223761923 wait complete. 2024-04-21T10:30:31.173 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-21T10:30:31.421 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:31 smithi026 ceph-mon[27965]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:31 smithi026 ceph-mon[27965]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:31 smithi169 ceph-mon[31469]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:31 smithi169 ceph-mon[31469]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:31.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:31 smithi190 ceph-mon[31423]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:30:31.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:31 smithi190 ceph-mon[31423]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.085+0000 7f6d48202700 1 -- 172.21.15.26:0/709223681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d40102ba0 msgr2=0x7f6d40103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.085+0000 7f6d48202700 1 --2- 172.21.15.26:0/709223681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d40102ba0 0x7f6d40103000 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d340099c0 tx=0x7f6d34009cd0 comp rx=0 tx=0).stop 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.085+0000 7f6d48202700 1 -- 172.21.15.26:0/709223681 shutdown_connections 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.085+0000 7f6d48202700 1 --2- 172.21.15.26:0/709223681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d401019a0 0x7f6d40101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.085+0000 7f6d48202700 1 --2- 172.21.15.26:0/709223681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d40103540 0x7f6d40109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.085+0000 7f6d48202700 1 --2- 172.21.15.26:0/709223681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d40102ba0 0x7f6d40103000 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.085+0000 7f6d48202700 1 -- 172.21.15.26:0/709223681 >> 172.21.15.26:0/709223681 conn(0x7f6d400fcf40 msgr2=0x7f6d400ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.085+0000 7f6d48202700 1 -- 172.21.15.26:0/709223681 shutdown_connections 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.085+0000 7f6d48202700 1 -- 172.21.15.26:0/709223681 wait complete. 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d48202700 1 Processor -- start 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d48202700 1 -- start start 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d48202700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d401019a0 0x7f6d4006b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d48202700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d40102ba0 0x7f6d4019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d48202700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d40103540 0x7f6d4019ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d48202700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4010d440 con 0x7f6d40103540 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d48202700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4010d2c0 con 0x7f6d40102ba0 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d48202700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4010d5c0 con 0x7f6d401019a0 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d4679f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d40103540 0x7f6d4019ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d4679f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d40103540 0x7f6d4019ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53054/0 (socket says 172.21.15.26:53054) 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d4679f700 1 -- 172.21.15.26:0/3604967751 learned_addr learned my addr 172.21.15.26:0/3604967751 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d45f9e700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d401019a0 0x7f6d4006b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d4579d700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d40102ba0 0x7f6d4019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d4679f700 1 -- 172.21.15.26:0/3604967751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d401019a0 msgr2=0x7f6d4006b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d4679f700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d401019a0 0x7f6d4006b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d4679f700 1 -- 172.21.15.26:0/3604967751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d40102ba0 msgr2=0x7f6d4019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d4679f700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d40102ba0 0x7f6d4019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.086+0000 7f6d4679f700 1 -- 172.21.15.26:0/3604967751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d401a08a0 con 0x7f6d40103540 2024-04-21T10:30:33.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.087+0000 7f6d4579d700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d40102ba0 0x7f6d4019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:33.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.087+0000 7f6d4679f700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d40103540 0x7f6d4019ffd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800d980 tx=0x7f6d3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:33.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.087+0000 7f6d3effd700 1 -- 172.21.15.26:0/3604967751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38017070 con 0x7f6d40103540 2024-04-21T10:30:33.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.087+0000 7f6d3effd700 1 -- 172.21.15.26:0/3604967751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d38005020 con 0x7f6d40103540 2024-04-21T10:30:33.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.087+0000 7f6d48202700 1 -- 172.21.15.26:0/3604967751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d401a0b90 con 0x7f6d40103540 2024-04-21T10:30:33.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.087+0000 7f6d3effd700 1 -- 172.21.15.26:0/3604967751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d380099b0 con 0x7f6d40103540 2024-04-21T10:30:33.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.087+0000 7f6d48202700 1 -- 172.21.15.26:0/3604967751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d40197760 con 0x7f6d40103540 2024-04-21T10:30:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.088+0000 7f6d48202700 1 -- 172.21.15.26:0/3604967751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d40069730 con 0x7f6d40103540 2024-04-21T10:30:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.089+0000 7f6d3effd700 1 -- 172.21.15.26:0/3604967751 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6d38004a30 con 0x7f6d40103540 2024-04-21T10:30:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.090+0000 7f6d3effd700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6d280a3cc0 0x7f6d280a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.090+0000 7f6d45f9e700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6d280a3cc0 0x7f6d280a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.090+0000 7f6d3effd700 1 -- 172.21.15.26:0/3604967751 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f6d380d05f0 con 0x7f6d40103540 2024-04-21T10:30:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.091+0000 7f6d45f9e700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6d280a3cc0 0x7f6d280a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009550 tx=0x7f6d300091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.095+0000 7f6d3effd700 1 -- 172.21.15.26:0/3604967751 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d38020020 con 0x7f6d40103540 2024-04-21T10:30:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.311+0000 7f6d48202700 1 -- 172.21.15.26:0/3604967751 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f6d40062e80 con 0x7f6d280a3cc0 2024-04-21T10:30:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.317+0000 7f6d3effd700 1 -- 172.21.15.26:0/3604967751 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f6d40062e80 con 0x7f6d280a3cc0 2024-04-21T10:30:33.317 INFO:teuthology.orchestra.run.smithi026.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-21T10:30:33.317 INFO:teuthology.orchestra.run.smithi026.stdout:mds.all 12/12 30s ago 2m smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi169=b;smithi169=e;smithi169=h;smithi169=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2024-04-21T10:30:33.317 INFO:teuthology.orchestra.run.smithi026.stdout:mgr 3/3 30s ago 7m smithi026=x;smithi169=y;smithi190=z;count:3 2024-04-21T10:30:33.317 INFO:teuthology.orchestra.run.smithi026.stdout:mon 3/3 30s ago 7m smithi026:172.21.15.26=a;smithi169:172.21.15.169=b;smithi190:172.21.15.190=c;count:3 2024-04-21T10:30:33.317 INFO:teuthology.orchestra.run.smithi026.stdout:osd 12 30s ago - 2024-04-21T10:30:33.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.319+0000 7f6d48202700 1 -- 172.21.15.26:0/3604967751 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6d280a3cc0 msgr2=0x7f6d280a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:33.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.319+0000 7f6d48202700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6d280a3cc0 0x7f6d280a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009550 tx=0x7f6d300091b0 comp rx=0 tx=0).stop 2024-04-21T10:30:33.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.319+0000 7f6d48202700 1 -- 172.21.15.26:0/3604967751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d40103540 msgr2=0x7f6d4019ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:33.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.319+0000 7f6d48202700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d40103540 0x7f6d4019ffd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800d980 tx=0x7f6d3800dc90 comp rx=0 tx=0).stop 2024-04-21T10:30:33.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.319+0000 7f6d48202700 1 -- 172.21.15.26:0/3604967751 shutdown_connections 2024-04-21T10:30:33.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.319+0000 7f6d48202700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f6d280a3cc0 0x7f6d280a6180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:33.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.319+0000 7f6d48202700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d40103540 0x7f6d4019ffd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:33.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.319+0000 7f6d48202700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d401019a0 0x7f6d4006b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:33.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.319+0000 7f6d48202700 1 --2- 172.21.15.26:0/3604967751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d40102ba0 0x7f6d4019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:33.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.319+0000 7f6d48202700 1 -- 172.21.15.26:0/3604967751 >> 172.21.15.26:0/3604967751 conn(0x7f6d400fcf40 msgr2=0x7f6d40105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:33.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.319+0000 7f6d48202700 1 -- 172.21.15.26:0/3604967751 shutdown_connections 2024-04-21T10:30:33.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:33.319+0000 7f6d48202700 1 -- 172.21.15.26:0/3604967751 wait complete. 2024-04-21T10:30:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:33 smithi026 ceph-mon[27965]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-21T10:30:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:33 smithi026 ceph-mon[27965]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-21T10:30:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:33 smithi169 ceph-mon[31469]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-21T10:30:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:33 smithi169 ceph-mon[31469]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-21T10:30:33.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:33 smithi190 ceph-mon[31423]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-21T10:30:33.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:33 smithi190 ceph-mon[31423]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-21T10:30:33.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-21T10:30:34.212 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:34.602 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:34 smithi026 ceph-mon[27965]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:34.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:34 smithi169 ceph-mon[31469]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:34.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:34 smithi190 ceph-mon[31423]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:35.679 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:35 smithi026 ceph-mon[27965]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-21T10:30:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:35 smithi169 ceph-mon[31469]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-21T10:30:35.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:35 smithi190 ceph-mon[31423]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-21T10:30:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.901+0000 7f8ac40e7700 1 -- 172.21.15.26:0/3350753167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1035f0 msgr2=0x7f8abc1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.901+0000 7f8ac40e7700 1 --2- 172.21.15.26:0/3350753167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1035f0 0x7f8abc1059e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009990 tx=0x7f8aac009ca0 comp rx=0 tx=0).stop 2024-04-21T10:30:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.901+0000 7f8ac40e7700 1 -- 172.21.15.26:0/3350753167 shutdown_connections 2024-04-21T10:30:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.901+0000 7f8ac40e7700 1 --2- 172.21.15.26:0/3350753167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8abc108850 0x7f8abc10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.901+0000 7f8ac40e7700 1 --2- 172.21.15.26:0/3350753167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc1035f0 0x7f8abc1059e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.901+0000 7f8ac40e7700 1 --2- 172.21.15.26:0/3350753167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8abc105f20 0x7f8abc108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.901+0000 7f8ac40e7700 1 -- 172.21.15.26:0/3350753167 >> 172.21.15.26:0/3350753167 conn(0x7f8abc0fcf60 msgr2=0x7f8abc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.901+0000 7f8ac40e7700 1 -- 172.21.15.26:0/3350753167 shutdown_connections 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.902+0000 7f8ac40e7700 1 -- 172.21.15.26:0/3350753167 wait complete. 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.902+0000 7f8ac40e7700 1 Processor -- start 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.902+0000 7f8ac40e7700 1 -- start start 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.902+0000 7f8ac40e7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8abc1035f0 0x7f8abc198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.902+0000 7f8ac40e7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8abc105f20 0x7f8abc199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.902+0000 7f8ac40e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc108850 0x7f8abc1a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.902+0000 7f8ac40e7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc10e510 con 0x7f8abc108850 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.902+0000 7f8ac40e7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc10e390 con 0x7f8abc1035f0 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.902+0000 7f8ac40e7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc10e690 con 0x7f8abc105f20 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.902+0000 7f8ac2684700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc108850 0x7f8abc1a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ac2684700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc108850 0x7f8abc1a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50602/0 (socket says 172.21.15.26:50602) 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ac2684700 1 -- 172.21.15.26:0/600864029 learned_addr learned my addr 172.21.15.26:0/600864029 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ac1682700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8abc105f20 0x7f8abc199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ac1e83700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8abc1035f0 0x7f8abc198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ac2684700 1 -- 172.21.15.26:0/600864029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8abc105f20 msgr2=0x7f8abc199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ac2684700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8abc105f20 0x7f8abc199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ac2684700 1 -- 172.21.15.26:0/600864029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8abc1035f0 msgr2=0x7f8abc198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ac2684700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8abc1035f0 0x7f8abc198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ac2684700 1 -- 172.21.15.26:0/600864029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc1a0880 con 0x7f8abc108850 2024-04-21T10:30:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ac2684700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc108850 0x7f8abc1a01b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4007ee0 tx=0x7f8ab400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ab2ffd700 1 -- 172.21.15.26:0/600864029 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4004bb0 con 0x7f8abc108850 2024-04-21T10:30:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ac40e7700 1 -- 172.21.15.26:0/600864029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc1a0b70 con 0x7f8abc108850 2024-04-21T10:30:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ab2ffd700 1 -- 172.21.15.26:0/600864029 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab4004d50 con 0x7f8abc108850 2024-04-21T10:30:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ab2ffd700 1 -- 172.21.15.26:0/600864029 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4005920 con 0x7f8abc108850 2024-04-21T10:30:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.903+0000 7f8ac40e7700 1 -- 172.21.15.26:0/600864029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc1a1060 con 0x7f8abc108850 2024-04-21T10:30:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.904+0000 7f8ac40e7700 1 -- 172.21.15.26:0/600864029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8abc069730 con 0x7f8abc108850 2024-04-21T10:30:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.905+0000 7f8ab2ffd700 1 -- 172.21.15.26:0/600864029 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8ab401a020 con 0x7f8abc108850 2024-04-21T10:30:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.906+0000 7f8ab2ffd700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8aa80a3c40 0x7f8aa80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.906+0000 7f8ac1e83700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8aa80a3c40 0x7f8aa80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.907+0000 7f8ab2ffd700 1 -- 172.21.15.26:0/600864029 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f8ab40d0190 con 0x7f8abc108850 2024-04-21T10:30:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.907+0000 7f8ac1e83700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8aa80a3c40 0x7f8aa80a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0038a0 tx=0x7f8aac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:35.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:35.910+0000 7f8ab2ffd700 1 -- 172.21.15.26:0/600864029 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ab4010070 con 0x7f8abc108850 2024-04-21T10:30:36.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.125+0000 7f8ac40e7700 1 -- 172.21.15.26:0/600864029 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f8abc062e80 con 0x7f8aa80a3c40 2024-04-21T10:30:36.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.126+0000 7f8ab2ffd700 1 -- 172.21.15.26:0/600864029 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f8abc062e80 con 0x7f8aa80a3c40 2024-04-21T10:30:36.126 INFO:teuthology.orchestra.run.smithi026.stdout:HOST ADDR LABELS STATUS 2024-04-21T10:30:36.126 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026 172.21.15.26 2024-04-21T10:30:36.126 INFO:teuthology.orchestra.run.smithi026.stdout:smithi169 172.21.15.169 2024-04-21T10:30:36.126 INFO:teuthology.orchestra.run.smithi026.stdout:smithi190 172.21.15.190 2024-04-21T10:30:36.126 INFO:teuthology.orchestra.run.smithi026.stdout:3 hosts in cluster 2024-04-21T10:30:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.128+0000 7f8ac40e7700 1 -- 172.21.15.26:0/600864029 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8aa80a3c40 msgr2=0x7f8aa80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.128+0000 7f8ac40e7700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8aa80a3c40 0x7f8aa80a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0038a0 tx=0x7f8aac005fb0 comp rx=0 tx=0).stop 2024-04-21T10:30:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.128+0000 7f8ac40e7700 1 -- 172.21.15.26:0/600864029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc108850 msgr2=0x7f8abc1a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.128+0000 7f8ac40e7700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc108850 0x7f8abc1a01b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4007ee0 tx=0x7f8ab400d3b0 comp rx=0 tx=0).stop 2024-04-21T10:30:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.128+0000 7f8ac40e7700 1 -- 172.21.15.26:0/600864029 shutdown_connections 2024-04-21T10:30:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.128+0000 7f8ac40e7700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8aa80a3c40 0x7f8aa80a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.128+0000 7f8ac40e7700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8abc1035f0 0x7f8abc198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.128+0000 7f8ac40e7700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8abc108850 0x7f8abc1a01b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.128+0000 7f8ac40e7700 1 --2- 172.21.15.26:0/600864029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8abc105f20 0x7f8abc199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.128+0000 7f8ac40e7700 1 -- 172.21.15.26:0/600864029 >> 172.21.15.26:0/600864029 conn(0x7f8abc0fcf60 msgr2=0x7f8abc104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.129+0000 7f8ac40e7700 1 -- 172.21.15.26:0/600864029 shutdown_connections 2024-04-21T10:30:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:36.129+0000 7f8ac40e7700 1 -- 172.21.15.26:0/600864029 wait complete. 2024-04-21T10:30:36.785 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-21T10:30:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:37.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:37 smithi026 ceph-mon[27965]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:37.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:37 smithi026 ceph-mon[27965]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-21T10:30:37.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:37 smithi169 ceph-mon[31469]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:37.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:37 smithi169 ceph-mon[31469]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-21T10:30:37.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:37 smithi190 ceph-mon[31423]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:37.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:37 smithi190 ceph-mon[31423]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-21T10:30:38.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.667+0000 7fa84a7da700 1 -- 172.21.15.26:0/3998026526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844101130 msgr2=0x7fa844101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:38.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.667+0000 7fa84a7da700 1 --2- 172.21.15.26:0/3998026526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844101130 0x7fa844101550 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa8340099c0 tx=0x7fa834009cd0 comp rx=0 tx=0).stop 2024-04-21T10:30:38.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.667+0000 7fa84a7da700 1 -- 172.21.15.26:0/3998026526 shutdown_connections 2024-04-21T10:30:38.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.667+0000 7fa84a7da700 1 --2- 172.21.15.26:0/3998026526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa844102cd0 0x7fa844109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:38.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.667+0000 7fa84a7da700 1 --2- 172.21.15.26:0/3998026526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844101130 0x7fa844101550 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:38.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.667+0000 7fa84a7da700 1 --2- 172.21.15.26:0/3998026526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa844102330 0x7fa844102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.667+0000 7fa84a7da700 1 -- 172.21.15.26:0/3998026526 >> 172.21.15.26:0/3998026526 conn(0x7fa8440fc6b0 msgr2=0x7fa8440feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.667+0000 7fa84a7da700 1 -- 172.21.15.26:0/3998026526 shutdown_connections 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.667+0000 7fa84a7da700 1 -- 172.21.15.26:0/3998026526 wait complete. 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.668+0000 7fa84a7da700 1 Processor -- start 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.668+0000 7fa84a7da700 1 -- start start 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.668+0000 7fa84a7da700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa844101130 0x7fa84410ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.668+0000 7fa84a7da700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844102330 0x7fa84419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa84a7da700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa844102cd0 0x7fa84419ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa84a7da700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa84410d480 con 0x7fa844102330 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa84a7da700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa84410d300 con 0x7fa844101130 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa84a7da700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa84410d600 con 0x7fa844102cd0 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa8437fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844102330 0x7fa84419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:38.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa8437fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844102330 0x7fa84419d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50622/0 (socket says 172.21.15.26:50622) 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa8437fe700 1 -- 172.21.15.26:0/2743790063 learned_addr learned my addr 172.21.15.26:0/2743790063 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa848d77700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa844102cd0 0x7fa84419ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa8437fe700 1 -- 172.21.15.26:0/2743790063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa844102cd0 msgr2=0x7fa84419ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa8437fe700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa844102cd0 0x7fa84419ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa843fff700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa844101130 0x7fa84410ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa8437fe700 1 -- 172.21.15.26:0/2743790063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa844101130 msgr2=0x7fa84410ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa8437fe700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa844101130 0x7fa84410ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa8437fe700 1 -- 172.21.15.26:0/2743790063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8441a0890 con 0x7fa844102330 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa843fff700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa844101130 0x7fa84410ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa8437fe700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844102330 0x7fa84419d930 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa83800d950 tx=0x7fa83800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa8417fa700 1 -- 172.21.15.26:0/2743790063 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838016070 con 0x7fa844102330 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa8417fa700 1 -- 172.21.15.26:0/2743790063 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa838010460 con 0x7fa844102330 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa84a7da700 1 -- 172.21.15.26:0/2743790063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8441a0b80 con 0x7fa844102330 2024-04-21T10:30:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa8417fa700 1 -- 172.21.15.26:0/2743790063 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83800f620 con 0x7fa844102330 2024-04-21T10:30:38.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.669+0000 7fa84a7da700 1 -- 172.21.15.26:0/2743790063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8441a10c0 con 0x7fa844102330 2024-04-21T10:30:38.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.670+0000 7fa84a7da700 1 -- 172.21.15.26:0/2743790063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa844069730 con 0x7fa844102330 2024-04-21T10:30:38.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.672+0000 7fa8417fa700 1 -- 172.21.15.26:0/2743790063 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa838019070 con 0x7fa844102330 2024-04-21T10:30:38.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.672+0000 7fa8417fa700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa82c0a3c40 0x7fa82c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:38.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.672+0000 7fa843fff700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa82c0a3c40 0x7fa82c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:38.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.672+0000 7fa8417fa700 1 -- 172.21.15.26:0/2743790063 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fa8380d0000 con 0x7fa844102330 2024-04-21T10:30:38.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.673+0000 7fa843fff700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa82c0a3c40 0x7fa82c0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa834006010 tx=0x7fa834005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:38.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.676+0000 7fa8417fa700 1 -- 172.21.15.26:0/2743790063 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa83809ca70 con 0x7fa844102330 2024-04-21T10:30:38.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.892+0000 7fa84a7da700 1 -- 172.21.15.26:0/2743790063 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa844062e80 con 0x7fa82c0a3c40 2024-04-21T10:30:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.894+0000 7fa8417fa700 1 -- 172.21.15.26:0/2743790063 <== mgr.14152 v2:172.21.15.26:6800/72610606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa844062e80 con 0x7fa82c0a3c40 2024-04-21T10:30:38.894 INFO:teuthology.orchestra.run.smithi026.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-21T10:30:38.894 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760020480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-21T10:30:38.894 INFO:teuthology.orchestra.run.smithi026.stdout:smithi169 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300LN400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-21T10:30:38.894 INFO:teuthology.orchestra.run.smithi026.stdout:smithi190 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT63210012400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-21T10:30:38.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.896+0000 7fa84a7da700 1 -- 172.21.15.26:0/2743790063 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa82c0a3c40 msgr2=0x7fa82c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:38.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.896+0000 7fa84a7da700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa82c0a3c40 0x7fa82c0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa834006010 tx=0x7fa834005e80 comp rx=0 tx=0).stop 2024-04-21T10:30:38.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.897+0000 7fa84a7da700 1 -- 172.21.15.26:0/2743790063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844102330 msgr2=0x7fa84419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:38.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.897+0000 7fa84a7da700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844102330 0x7fa84419d930 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa83800d950 tx=0x7fa83800dd10 comp rx=0 tx=0).stop 2024-04-21T10:30:38.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.897+0000 7fa84a7da700 1 -- 172.21.15.26:0/2743790063 shutdown_connections 2024-04-21T10:30:38.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.897+0000 7fa84a7da700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fa82c0a3c40 0x7fa82c0a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:38.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.897+0000 7fa84a7da700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa844101130 0x7fa84410ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:38.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.897+0000 7fa84a7da700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa844102cd0 0x7fa84419ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:38.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.897+0000 7fa84a7da700 1 --2- 172.21.15.26:0/2743790063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844102330 0x7fa84419d930 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:38.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.897+0000 7fa84a7da700 1 -- 172.21.15.26:0/2743790063 >> 172.21.15.26:0/2743790063 conn(0x7fa8440fc6b0 msgr2=0x7fa8441056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:38.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.897+0000 7fa84a7da700 1 -- 172.21.15.26:0/2743790063 shutdown_connections 2024-04-21T10:30:38.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:38.897+0000 7fa84a7da700 1 -- 172.21.15.26:0/2743790063 wait complete. 2024-04-21T10:30:39.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:39 smithi026 ceph-mon[27965]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-21T10:30:39.594 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:39 smithi026 ceph-mon[27965]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:39.626 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-21T10:30:39.635 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-04-21T10:30:39.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-21T10:30:39.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:39 smithi169 ceph-mon[31469]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-21T10:30:39.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:39 smithi169 ceph-mon[31469]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:39.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:39 smithi190 ceph-mon[31423]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-21T10:30:39.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:39 smithi190 ceph-mon[31423]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:30:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:41.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.499+0000 7f3f8d967700 1 -- 172.21.15.26:0/1286198215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f88105e80 msgr2=0x7f3f88108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:41.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.499+0000 7f3f8d967700 1 --2- 172.21.15.26:0/1286198215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f88105e80 0x7f3f88108270 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009990 tx=0x7f3f7c009ca0 comp rx=0 tx=0).stop 2024-04-21T10:30:41.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.499+0000 7f3f8d967700 1 -- 172.21.15.26:0/1286198215 shutdown_connections 2024-04-21T10:30:41.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.499+0000 7f3f8d967700 1 --2- 172.21.15.26:0/1286198215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f881087b0 0x7f3f8810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:41.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.499+0000 7f3f8d967700 1 --2- 172.21.15.26:0/1286198215 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f88103550 0x7f3f88105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:41.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.499+0000 7f3f8d967700 1 --2- 172.21.15.26:0/1286198215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f88105e80 0x7f3f88108270 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:41.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.499+0000 7f3f8d967700 1 -- 172.21.15.26:0/1286198215 >> 172.21.15.26:0/1286198215 conn(0x7f3f880fcf60 msgr2=0x7f3f880ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:41.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.499+0000 7f3f8d967700 1 -- 172.21.15.26:0/1286198215 shutdown_connections 2024-04-21T10:30:41.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.499+0000 7f3f8d967700 1 -- 172.21.15.26:0/1286198215 wait complete. 2024-04-21T10:30:41.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.500+0000 7f3f8d967700 1 Processor -- start 2024-04-21T10:30:41.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.500+0000 7f3f8d967700 1 -- start start 2024-04-21T10:30:41.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.500+0000 7f3f8d967700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f88103550 0x7f3f88198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:41.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.500+0000 7f3f8d967700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f88105e80 0x7f3f88199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:41.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.500+0000 7f3f8d967700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f881087b0 0x7f3f881a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:41.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.500+0000 7f3f8d967700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8810e550 con 0x7f3f88103550 2024-04-21T10:30:41.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.500+0000 7f3f8d967700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8810e3d0 con 0x7f3f881087b0 2024-04-21T10:30:41.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.500+0000 7f3f8d967700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8810e6d0 con 0x7f3f88105e80 2024-04-21T10:30:41.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.500+0000 7f3f86ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f88103550 0x7f3f88198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:41.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.500+0000 7f3f86ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f88103550 0x7f3f88198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50646/0 (socket says 172.21.15.26:50646) 2024-04-21T10:30:41.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.500+0000 7f3f86ffd700 1 -- 172.21.15.26:0/2401143754 learned_addr learned my addr 172.21.15.26:0/2401143754 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:41.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f877fe700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f881087b0 0x7f3f881a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:41.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f867fc700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f88105e80 0x7f3f88199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:41.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f86ffd700 1 -- 172.21.15.26:0/2401143754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f88105e80 msgr2=0x7f3f88199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:41.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f86ffd700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f88105e80 0x7f3f88199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:41.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f86ffd700 1 -- 172.21.15.26:0/2401143754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f881087b0 msgr2=0x7f3f881a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:41.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f86ffd700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f881087b0 0x7f3f881a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:41.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f86ffd700 1 -- 172.21.15.26:0/2401143754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f881a0870 con 0x7f3f88103550 2024-04-21T10:30:41.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f86ffd700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f88103550 0x7f3f88198c20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c920 tx=0x7f3f7800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:41.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f8c965700 1 -- 172.21.15.26:0/2401143754 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78004a30 con 0x7f3f88103550 2024-04-21T10:30:41.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f8d967700 1 -- 172.21.15.26:0/2401143754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f881a0b60 con 0x7f3f88103550 2024-04-21T10:30:41.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f8c965700 1 -- 172.21.15.26:0/2401143754 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f780054d0 con 0x7f3f88103550 2024-04-21T10:30:41.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f8c965700 1 -- 172.21.15.26:0/2401143754 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78007bd0 con 0x7f3f88103550 2024-04-21T10:30:41.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.501+0000 7f3f8d967700 1 -- 172.21.15.26:0/2401143754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f881a10f0 con 0x7f3f88103550 2024-04-21T10:30:41.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.502+0000 7f3f8d967700 1 -- 172.21.15.26:0/2401143754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f88069730 con 0x7f3f88103550 2024-04-21T10:30:41.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.503+0000 7f3f8c965700 1 -- 172.21.15.26:0/2401143754 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3f78004bd0 con 0x7f3f88103550 2024-04-21T10:30:41.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.504+0000 7f3f8c965700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3f700a3c40 0x7f3f700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:41.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.504+0000 7f3f8c965700 1 -- 172.21.15.26:0/2401143754 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f3f780cff80 con 0x7f3f88103550 2024-04-21T10:30:41.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.504+0000 7f3f867fc700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3f700a3c40 0x7f3f700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:41.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.504+0000 7f3f867fc700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3f700a3c40 0x7f3f700a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00b5c0 tx=0x7f3f7c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:41.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.509+0000 7f3f8c965700 1 -- 172.21.15.26:0/2401143754 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f78014030 con 0x7f3f88103550 2024-04-21T10:30:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:41 smithi026 ceph-mon[27965]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-21T10:30:41.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:41 smithi169 ceph-mon[31469]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-21T10:30:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.764+0000 7f3f8d967700 1 -- 172.21.15.26:0/2401143754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3f88063650 con 0x7f3f88103550 2024-04-21T10:30:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.764+0000 7f3f8c965700 1 -- 172.21.15.26:0/2401143754 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+3058 (secure 0 0 0) 0x7f3f7809c8f0 con 0x7f3f88103550 2024-04-21T10:30:41.764 INFO:teuthology.orchestra.run.smithi026.stdout:e20 2024-04-21T10:30:41.764 INFO:teuthology.orchestra.run.smithi026.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T10:30:41.764 INFO:teuthology.orchestra.run.smithi026.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T10:30:41.764 INFO:teuthology.orchestra.run.smithi026.stdout:legacy client fscid: 1 2024-04-21T10:30:41.764 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:41.764 INFO:teuthology.orchestra.run.smithi026.stdout:Filesystem 'cephfs' (1) 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:fs_name cephfs 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 20 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-04-21T10:29:01.278914+0000 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:modified 2024-04-21T10:29:05.806617+0000 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:tableserver 0 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:root 0 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:session_timeout 300 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:session_autoclose 300 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:max_file_size 1099511627776 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:max_xattr_size 65536 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:required_client_features {} 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:last_failure 0 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:last_failure_osd_epoch 0 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:max_mds 3 2024-04-21T10:30:41.765 INFO:teuthology.orchestra.run.smithi026.stdout:in 0,1,2 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:up {0=24496,1=24488,2=24478} 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:failed 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:damaged 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:stopped 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:data_pools [3] 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:metadata_pool 2 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:inline_data disabled 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:balancer 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:bal_rank_mask -1 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:standby_count_wanted 0 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:qdb_cluster leader: 24496 members: 24478,24488,24496 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.i{0:24496} state up:active seq 5 addr [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.f{0:24466} state up:standby-replay seq 1 addr [v2:172.21.15.190:6834/1573483809,v1:172.21.15.190:6835/1573483809] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b{1:24488} state up:active seq 3 addr [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.h{1:24449} state up:standby-replay seq 1 addr [v2:172.21.15.169:6836/2325983283,v1:172.21.15.169:6837/2325983283] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T10:30:41.766 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c{2:24478} state up:active seq 7 addr [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T10:30:41.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.e{2:24428} state up:standby-replay seq 1 addr [v2:172.21.15.169:6834/1594119566,v1:172.21.15.169:6835/1594119566] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T10:30:41.767 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:41.767 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:41.767 INFO:teuthology.orchestra.run.smithi026.stdout:Standby daemons: 2024-04-21T10:30:41.767 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:41.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.j{-1:14547} state up:standby seq 1 addr [v2:172.21.15.26:6834/2165061396,v1:172.21.15.26:6835/2165061396] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T10:30:41.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.26:6836/4049177866,v1:172.21.15.26:6837/4049177866] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T10:30:41.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.g{-1:14631} state up:standby seq 1 addr [v2:172.21.15.26:6838/3108850523,v1:172.21.15.26:6839/3108850523] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T10:30:41.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a{-1:14643} state up:standby seq 1 addr [v2:172.21.15.26:6840/1075023600,v1:172.21.15.26:6841/1075023600] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T10:30:41.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.k{-1:24398} state up:standby seq 1 addr [v2:172.21.15.169:6832/2779252400,v1:172.21.15.169:6833/2779252400] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T10:30:41.767 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.l{-1:24415} state up:standby seq 1 addr [v2:172.21.15.190:6832/2081925034,v1:172.21.15.190:6833/2081925034] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T10:30:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.766+0000 7f3f8d967700 1 -- 172.21.15.26:0/2401143754 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3f700a3c40 msgr2=0x7f3f700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.766+0000 7f3f8d967700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3f700a3c40 0x7f3f700a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00b5c0 tx=0x7f3f7c005fb0 comp rx=0 tx=0).stop 2024-04-21T10:30:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.766+0000 7f3f8d967700 1 -- 172.21.15.26:0/2401143754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f88103550 msgr2=0x7f3f88198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.766+0000 7f3f8d967700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f88103550 0x7f3f88198c20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c920 tx=0x7f3f7800cc30 comp rx=0 tx=0).stop 2024-04-21T10:30:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.766+0000 7f3f8d967700 1 -- 172.21.15.26:0/2401143754 shutdown_connections 2024-04-21T10:30:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.766+0000 7f3f8d967700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f3f700a3c40 0x7f3f700a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.766+0000 7f3f8d967700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f881087b0 0x7f3f881a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.766+0000 7f3f8d967700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f88103550 0x7f3f88198c20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.766+0000 7f3f8d967700 1 --2- 172.21.15.26:0/2401143754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f88105e80 0x7f3f88199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.766+0000 7f3f8d967700 1 -- 172.21.15.26:0/2401143754 >> 172.21.15.26:0/2401143754 conn(0x7f3f880fcf60 msgr2=0x7f3f88104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.767+0000 7f3f8d967700 1 -- 172.21.15.26:0/2401143754 shutdown_connections 2024-04-21T10:30:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:41.767+0000 7f3f8d967700 1 -- 172.21.15.26:0/2401143754 wait complete. 2024-04-21T10:30:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 20 2024-04-21T10:30:41.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:41 smithi190 ceph-mon[31423]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-21T10:30:42.535 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-21T10:30:42.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:42 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2401143754' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T10:30:42.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:42 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2401143754' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T10:30:42.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:42 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2401143754' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T10:30:42.788 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:43 smithi026 ceph-mon[27965]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-21T10:30:43.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:43 smithi169 ceph-mon[31469]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-21T10:30:43.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:43 smithi190 ceph-mon[31423]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-21T10:30:44.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.649+0000 7f68c99f2700 1 -- 172.21.15.26:0/3534727526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68c4100d50 msgr2=0x7f68c4101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.649+0000 7f68c99f2700 1 --2- 172.21.15.26:0/3534727526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68c4100d50 0x7f68c4101170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009990 tx=0x7f68b4009ca0 comp rx=0 tx=0).stop 2024-04-21T10:30:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.649+0000 7f68c99f2700 1 -- 172.21.15.26:0/3534727526 shutdown_connections 2024-04-21T10:30:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.649+0000 7f68c99f2700 1 --2- 172.21.15.26:0/3534727526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c41046d0 0x7f68c410af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.649+0000 7f68c99f2700 1 --2- 172.21.15.26:0/3534727526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68c4100d50 0x7f68c4101170 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.649+0000 7f68c99f2700 1 --2- 172.21.15.26:0/3534727526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68c4103d30 0x7f68c4104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.649+0000 7f68c99f2700 1 -- 172.21.15.26:0/3534727526 >> 172.21.15.26:0/3534727526 conn(0x7f68c40fc6d0 msgr2=0x7f68c40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.649+0000 7f68c99f2700 1 -- 172.21.15.26:0/3534727526 shutdown_connections 2024-04-21T10:30:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.649+0000 7f68c99f2700 1 -- 172.21.15.26:0/3534727526 wait complete. 2024-04-21T10:30:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c99f2700 1 Processor -- start 2024-04-21T10:30:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c99f2700 1 -- start start 2024-04-21T10:30:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c99f2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68c4100d50 0x7f68c4198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c99f2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c4103d30 0x7f68c4199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c99f2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68c41046d0 0x7f68c41a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c99f2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f68c410e4f0 con 0x7f68c4100d50 2024-04-21T10:30:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c99f2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f68c410e370 con 0x7f68c41046d0 2024-04-21T10:30:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c99f2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f68c410e670 con 0x7f68c4103d30 2024-04-21T10:30:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68c4100d50 0x7f68c4198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68c4100d50 0x7f68c4198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58098/0 (socket says 172.21.15.26:58098) 2024-04-21T10:30:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c2ffd700 1 -- 172.21.15.26:0/1762660720 learned_addr learned my addr 172.21.15.26:0/1762660720 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c37fe700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68c41046d0 0x7f68c41a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.650+0000 7f68c27fc700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c4103d30 0x7f68c4199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.651+0000 7f68c2ffd700 1 -- 172.21.15.26:0/1762660720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c4103d30 msgr2=0x7f68c4199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.651+0000 7f68c2ffd700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c4103d30 0x7f68c4199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.651+0000 7f68c2ffd700 1 -- 172.21.15.26:0/1762660720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68c41046d0 msgr2=0x7f68c41a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.651+0000 7f68c2ffd700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68c41046d0 0x7f68c41a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.651+0000 7f68c2ffd700 1 -- 172.21.15.26:0/1762660720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c41a0830 con 0x7f68c4100d50 2024-04-21T10:30:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.651+0000 7f68c2ffd700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68c4100d50 0x7f68c4198bd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f68b40047b0 tx=0x7f68b4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.651+0000 7f68c89f0700 1 -- 172.21.15.26:0/1762660720 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4027070 con 0x7f68c4100d50 2024-04-21T10:30:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.651+0000 7f68c99f2700 1 -- 172.21.15.26:0/1762660720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c41a0ac0 con 0x7f68c4100d50 2024-04-21T10:30:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.651+0000 7f68c89f0700 1 -- 172.21.15.26:0/1762660720 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68b4004140 con 0x7f68c4100d50 2024-04-21T10:30:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.651+0000 7f68c89f0700 1 -- 172.21.15.26:0/1762660720 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4022e50 con 0x7f68c4100d50 2024-04-21T10:30:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.651+0000 7f68c99f2700 1 -- 172.21.15.26:0/1762660720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c41a0f10 con 0x7f68c4100d50 2024-04-21T10:30:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.652+0000 7f68c99f2700 1 -- 172.21.15.26:0/1762660720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c4069730 con 0x7f68c4100d50 2024-04-21T10:30:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.653+0000 7f68c89f0700 1 -- 172.21.15.26:0/1762660720 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f68b402a070 con 0x7f68c4100d50 2024-04-21T10:30:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.654+0000 7f68c89f0700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f68ac0a3de0 0x7f68ac0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.654+0000 7f68c27fc700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f68ac0a3de0 0x7f68ac0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:44.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.654+0000 7f68c89f0700 1 -- 172.21.15.26:0/1762660720 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f68b40d1400 con 0x7f68c4100d50 2024-04-21T10:30:44.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.655+0000 7f68c27fc700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f68ac0a3de0 0x7f68ac0a62a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f68b8005950 tx=0x7f68b80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:44.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.658+0000 7f68c89f0700 1 -- 172.21.15.26:0/1762660720 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68b4013070 con 0x7f68c4100d50 2024-04-21T10:30:44.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.865+0000 7f68c99f2700 1 -- 172.21.15.26:0/1762660720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f68c402e1b0 con 0x7f68c4100d50 2024-04-21T10:30:44.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.866+0000 7f68c89f0700 1 -- 172.21.15.26:0/1762660720 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5309 (secure 0 0 0) 0x7f68b409de60 con 0x7f68c4100d50 2024-04-21T10:30:44.865 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 89 2024-04-21T10:30:44.866 INFO:teuthology.orchestra.run.smithi026.stdout:fsid b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:30:44.866 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-04-21T10:21:08.637646+0000 2024-04-21T10:30:44.866 INFO:teuthology.orchestra.run.smithi026.stdout:modified 2024-04-21T10:30:03.297428+0000 2024-04-21T10:30:44.866 INFO:teuthology.orchestra.run.smithi026.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-21T10:30:44.866 INFO:teuthology.orchestra.run.smithi026.stdout:crush_version 26 2024-04-21T10:30:44.866 INFO:teuthology.orchestra.run.smithi026.stdout:full_ratio 0.95 2024-04-21T10:30:44.866 INFO:teuthology.orchestra.run.smithi026.stdout:backfillfull_ratio 0.9 2024-04-21T10:30:44.866 INFO:teuthology.orchestra.run.smithi026.stdout:nearfull_ratio 0.85 2024-04-21T10:30:44.866 INFO:teuthology.orchestra.run.smithi026.stdout:require_min_compat_client luminous 2024-04-21T10:30:44.866 INFO:teuthology.orchestra.run.smithi026.stdout:min_compat_client luminous 2024-04-21T10:30:44.866 INFO:teuthology.orchestra.run.smithi026.stdout:require_osd_release squid 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:stretch_mode_enabled false 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:max_osd 12 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6802/3221939108,v1:172.21.15.26:6803/3221939108] [v2:172.21.15.26:6804/3221939108,v1:172.21.15.26:6805/3221939108] exists,up 6a8bc52a-346f-4a31-aad9-535e551ba41b 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6800/4294740313,v1:172.21.15.169:6801/4294740313] [v2:172.21.15.169:6802/4294740313,v1:172.21.15.169:6803/4294740313] exists,up ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6800/2878992539,v1:172.21.15.190:6801/2878992539] [v2:172.21.15.190:6802/2878992539,v1:172.21.15.190:6803/2878992539] exists,up e7906664-6ec7-46e9-a500-50bac1bc7674 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:osd.3 up in weight 1 up_from 26 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6810/2335886169,v1:172.21.15.26:6811/2335886169] [v2:172.21.15.26:6812/2335886169,v1:172.21.15.26:6813/2335886169] exists,up 62499ade-dc25-4d67-8e9f-5486c2298565 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:osd.4 up in weight 1 up_from 34 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6808/1708888642,v1:172.21.15.169:6809/1708888642] [v2:172.21.15.169:6810/1708888642,v1:172.21.15.169:6811/1708888642] exists,up a1b50fcf-cfcd-42a2-b246-eb9d736aa437 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:osd.5 up in weight 1 up_from 40 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6808/427467914,v1:172.21.15.190:6809/427467914] [v2:172.21.15.190:6810/427467914,v1:172.21.15.190:6811/427467914] exists,up 33e19659-9fe5-4fe5-92e4-167c782735b5 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:osd.6 up in weight 1 up_from 48 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6818/4211294337,v1:172.21.15.26:6819/4211294337] [v2:172.21.15.26:6820/4211294337,v1:172.21.15.26:6821/4211294337] exists,up 20cf2d5b-6cea-4f55-9b15-2feeea193919 2024-04-21T10:30:44.867 INFO:teuthology.orchestra.run.smithi026.stdout:osd.7 up in weight 1 up_from 53 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6816/16938185,v1:172.21.15.169:6817/16938185] [v2:172.21.15.169:6818/16938185,v1:172.21.15.169:6819/16938185] exists,up 184da4b1-7943-425d-8e97-c89af799e656 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:osd.8 up in weight 1 up_from 61 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6816/250015578,v1:172.21.15.190:6817/250015578] [v2:172.21.15.190:6818/250015578,v1:172.21.15.190:6819/250015578] exists,up 8da5499d-cd87-4182-87f7-6caffa3f5329 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:osd.9 up in weight 1 up_from 69 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6826/3856298247,v1:172.21.15.26:6827/3856298247] [v2:172.21.15.26:6828/3856298247,v1:172.21.15.26:6829/3856298247] exists,up 47e78d57-1332-4623-a4cf-dcd4d4af2945 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:osd.10 up in weight 1 up_from 74 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6824/3439107945,v1:172.21.15.169:6825/3439107945] [v2:172.21.15.169:6826/3439107945,v1:172.21.15.169:6827/3439107945] exists,up 11ff534c-dd49-446d-b532-f124b6a70b0b 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:osd.11 up in weight 1 up_from 82 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6824/1221661638,v1:172.21.15.190:6825/1221661638] [v2:172.21.15.190:6826/1221661638,v1:172.21.15.190:6827/1221661638] exists,up 2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.a [2,11] 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.10 [2,1,8,11] 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.1a [8,4] 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.1 [4,9] 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.2 [3,9] 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.1a [5,6] 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.21 [4,8] 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.24 [4,9] 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.28 [3,8] 2024-04-21T10:30:44.868 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/2138811732 expires 2024-04-22T10:22:02.083151+0000 2024-04-21T10:30:44.869 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/4216306847 expires 2024-04-22T10:22:02.083151+0000 2024-04-21T10:30:44.869 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6801/878369551 expires 2024-04-22T10:22:02.083151+0000 2024-04-21T10:30:44.869 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/2916501364 expires 2024-04-22T10:22:02.083151+0000 2024-04-21T10:30:44.869 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6800/3874640691 expires 2024-04-22T10:21:35.999459+0000 2024-04-21T10:30:44.869 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6800/878369551 expires 2024-04-22T10:22:02.083151+0000 2024-04-21T10:30:44.869 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/3866216235 expires 2024-04-22T10:21:35.999459+0000 2024-04-21T10:30:44.869 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6801/3874640691 expires 2024-04-22T10:21:35.999459+0000 2024-04-21T10:30:44.869 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/1439453622 expires 2024-04-22T10:21:35.999459+0000 2024-04-21T10:30:44.869 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/3848404931 expires 2024-04-22T10:21:35.999459+0000 2024-04-21T10:30:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.868+0000 7f68c99f2700 1 -- 172.21.15.26:0/1762660720 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f68ac0a3de0 msgr2=0x7f68ac0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.868+0000 7f68c99f2700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f68ac0a3de0 0x7f68ac0a62a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f68b8005950 tx=0x7f68b80058e0 comp rx=0 tx=0).stop 2024-04-21T10:30:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.868+0000 7f68c99f2700 1 -- 172.21.15.26:0/1762660720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68c4100d50 msgr2=0x7f68c4198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.868+0000 7f68c99f2700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68c4100d50 0x7f68c4198bd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f68b40047b0 tx=0x7f68b4005b00 comp rx=0 tx=0).stop 2024-04-21T10:30:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.868+0000 7f68c99f2700 1 -- 172.21.15.26:0/1762660720 shutdown_connections 2024-04-21T10:30:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.868+0000 7f68c99f2700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f68ac0a3de0 0x7f68ac0a62a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.868+0000 7f68c99f2700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68c41046d0 0x7f68c41a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.868+0000 7f68c99f2700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68c4100d50 0x7f68c4198bd0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.868+0000 7f68c99f2700 1 --2- 172.21.15.26:0/1762660720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c4103d30 0x7f68c4199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.868+0000 7f68c99f2700 1 -- 172.21.15.26:0/1762660720 >> 172.21.15.26:0/1762660720 conn(0x7f68c40fc6d0 msgr2=0x7f68c4107120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:44.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.868+0000 7f68c99f2700 1 -- 172.21.15.26:0/1762660720 shutdown_connections 2024-04-21T10:30:44.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:44.868+0000 7f68c99f2700 1 -- 172.21.15.26:0/1762660720 wait complete. 2024-04-21T10:30:45.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:45 smithi026 ceph-mon[27965]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:30:45.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:45 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1762660720' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T10:30:45.622 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-21T10:30:45.633 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:30:45.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:45 smithi169 ceph-mon[31469]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:30:45.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:45 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1762660720' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T10:30:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.743+0000 7f449aec1640 1 -- 172.21.15.26:0/238191086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4494103190 msgr2=0x7f4494103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.743+0000 7f449aec1640 1 --2- 172.21.15.26:0/238191086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4494103190 0x7f4494103610 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f447c009900 tx=0x7f447c02bc50 comp rx=0 tx=0).stop 2024-04-21T10:30:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.743+0000 7f449aec1640 1 -- 172.21.15.26:0/238191086 shutdown_connections 2024-04-21T10:30:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.743+0000 7f449aec1640 1 --2- 172.21.15.26:0/238191086 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4494103b50 0x7f449410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.743+0000 7f449aec1640 1 --2- 172.21.15.26:0/238191086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4494103190 0x7f4494103610 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.743+0000 7f449aec1640 1 --2- 172.21.15.26:0/238191086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4494101fa0 0x7f44941023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.743+0000 7f449aec1640 1 -- 172.21.15.26:0/238191086 >> 172.21.15.26:0/238191086 conn(0x7f44940fd530 msgr2=0x7f44940ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.743+0000 7f449aec1640 1 -- 172.21.15.26:0/238191086 shutdown_connections 2024-04-21T10:30:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.743+0000 7f449aec1640 1 -- 172.21.15.26:0/238191086 wait complete. 2024-04-21T10:30:45.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449aec1640 1 Processor -- start 2024-04-21T10:30:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449aec1640 1 -- start start 2024-04-21T10:30:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449aec1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4494101fa0 0x7f4494198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449aec1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4494103190 0x7f4494198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449aec1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4494103b50 0x7f449419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449aec1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f449410dae0 con 0x7f4494103b50 2024-04-21T10:30:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449aec1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f449410d960 con 0x7f4494103190 2024-04-21T10:30:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449aec1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f449410dc60 con 0x7f4494101fa0 2024-04-21T10:30:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449945e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4494103b50 0x7f449419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449945e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4494103b50 0x7f449419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58104/0 (socket says 172.21.15.26:58104) 2024-04-21T10:30:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449945e640 1 -- 172.21.15.26:0/1080159068 learned_addr learned my addr 172.21.15.26:0/1080159068 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:45.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f4493fff640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4494103190 0x7f4494198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f4498c5d640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4494101fa0 0x7f4494198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449945e640 1 -- 172.21.15.26:0/1080159068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4494101fa0 msgr2=0x7f4494198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449945e640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4494101fa0 0x7f4494198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449945e640 1 -- 172.21.15.26:0/1080159068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4494103190 msgr2=0x7f4494198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.744+0000 7f449945e640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4494103190 0x7f4494198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.745+0000 7f449945e640 1 -- 172.21.15.26:0/1080159068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449419fc70 con 0x7f4494103b50 2024-04-21T10:30:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.745+0000 7f449945e640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4494103b50 0x7f449419f600 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f448c00cce0 tx=0x7f448c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.745+0000 7f4491ffb640 1 -- 172.21.15.26:0/1080159068 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c00f450 con 0x7f4494103b50 2024-04-21T10:30:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.745+0000 7f4491ffb640 1 -- 172.21.15.26:0/1080159068 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f448c00faa0 con 0x7f4494103b50 2024-04-21T10:30:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.745+0000 7f4491ffb640 1 -- 172.21.15.26:0/1080159068 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c018b70 con 0x7f4494103b50 2024-04-21T10:30:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.745+0000 7f449aec1640 1 -- 172.21.15.26:0/1080159068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f449419ff60 con 0x7f4494103b50 2024-04-21T10:30:45.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.745+0000 7f449aec1640 1 -- 172.21.15.26:0/1080159068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44941a03d0 con 0x7f4494103b50 2024-04-21T10:30:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.746+0000 7f449aec1640 1 -- 172.21.15.26:0/1080159068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4494109260 con 0x7f4494103b50 2024-04-21T10:30:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.747+0000 7f4491ffb640 1 -- 172.21.15.26:0/1080159068 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f448c018d10 con 0x7f4494103b50 2024-04-21T10:30:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.748+0000 7f4491ffb640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f44800a3bb0 0x7f44800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.748+0000 7f4498c5d640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f44800a3bb0 0x7f44800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.748+0000 7f4491ffb640 1 -- 172.21.15.26:0/1080159068 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f448c0d0810 con 0x7f4494103b50 2024-04-21T10:30:45.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.748+0000 7f4498c5d640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f44800a3bb0 0x7f44800a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4494102ff0 tx=0x7f4484005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:45.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.753+0000 7f4491ffb640 1 -- 172.21.15.26:0/1080159068 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f448c014030 con 0x7f4494103b50 2024-04-21T10:30:45.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:45 smithi190 ceph-mon[31423]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:30:45.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:45 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1762660720' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T10:30:45.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.943+0000 7f449aec1640 1 -- 172.21.15.26:0/1080159068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4494062b50 con 0x7f4494103b50 2024-04-21T10:30:45.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.945+0000 7f4491ffb640 1 -- 172.21.15.26:0/1080159068 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22705 (secure 0 0 0) 0x7f448c09d180 con 0x7f4494103b50 2024-04-21T10:30:45.945 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:45.945 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:30:03.297428+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:30:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.947+0000 7f449aec1640 1 -- 172.21.15.26:0/1080159068 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f44800a3bb0 msgr2=0x7f44800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.947+0000 7f449aec1640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f44800a3bb0 0x7f44800a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4494102ff0 tx=0x7f4484005f90 comp rx=0 tx=0).stop 2024-04-21T10:30:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.947+0000 7f449aec1640 1 -- 172.21.15.26:0/1080159068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4494103b50 msgr2=0x7f449419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.947+0000 7f449aec1640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4494103b50 0x7f449419f600 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f448c00cce0 tx=0x7f448c00a620 comp rx=0 tx=0).stop 2024-04-21T10:30:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.947+0000 7f449aec1640 1 -- 172.21.15.26:0/1080159068 shutdown_connections 2024-04-21T10:30:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.947+0000 7f449aec1640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f44800a3bb0 0x7f44800a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:45.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.947+0000 7f449aec1640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4494103b50 0x7f449419f600 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:45.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.947+0000 7f449aec1640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4494103190 0x7f4494198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:45.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.947+0000 7f449aec1640 1 --2- 172.21.15.26:0/1080159068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4494101fa0 0x7f4494198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:45.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.947+0000 7f449aec1640 1 -- 172.21.15.26:0/1080159068 >> 172.21.15.26:0/1080159068 conn(0x7f44940fd530 msgr2=0x7f4494106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:45.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.947+0000 7f449aec1640 1 -- 172.21.15.26:0/1080159068 shutdown_connections 2024-04-21T10:30:45.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:45.947+0000 7f449aec1640 1 -- 172.21.15.26:0/1080159068 wait complete. 2024-04-21T10:30:45.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T10:24:28.176891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T10:28:59.417687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T10:29:00.234808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T10:30:45.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T10:30:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.064+0000 7fbf04881640 1 -- 172.21.15.26:0/382747814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf00102ac0 msgr2=0x7fbf00104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.064+0000 7fbf04881640 1 --2- 172.21.15.26:0/382747814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf00102ac0 0x7fbf00104eb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009ca0 tx=0x7fbef002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.064+0000 7fbf04881640 1 -- 172.21.15.26:0/382747814 shutdown_connections 2024-04-21T10:30:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.064+0000 7fbf04881640 1 --2- 172.21.15.26:0/382747814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf00107d20 0x7fbf0010a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.064+0000 7fbf04881640 1 --2- 172.21.15.26:0/382747814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf001053f0 0x7fbf001077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.065+0000 7fbf04881640 1 --2- 172.21.15.26:0/382747814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf00102ac0 0x7fbf00104eb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.065+0000 7fbf04881640 1 -- 172.21.15.26:0/382747814 >> 172.21.15.26:0/382747814 conn(0x7fbf000fc410 msgr2=0x7fbf000fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.065+0000 7fbf04881640 1 -- 172.21.15.26:0/382747814 shutdown_connections 2024-04-21T10:30:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.065+0000 7fbf04881640 1 -- 172.21.15.26:0/382747814 wait complete. 2024-04-21T10:30:46.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.065+0000 7fbf04881640 1 Processor -- start 2024-04-21T10:30:46.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.065+0000 7fbf04881640 1 -- start start 2024-04-21T10:30:46.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.065+0000 7fbf04881640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf00102ac0 0x7fbf00198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:46.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.065+0000 7fbf04881640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf001053f0 0x7fbf00198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:46.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbf04881640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf00107d20 0x7fbf0019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:46.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbf04881640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0010db40 con 0x7fbf00102ac0 2024-04-21T10:30:46.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbf04881640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0010d9c0 con 0x7fbf001053f0 2024-04-21T10:30:46.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbf04881640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0010dcc0 con 0x7fbf00107d20 2024-04-21T10:30:46.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbefe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf00102ac0 0x7fbf00198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:46.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbefe59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf00102ac0 0x7fbf00198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58126/0 (socket says 172.21.15.26:58126) 2024-04-21T10:30:46.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbefe59c640 1 -- 172.21.15.26:0/1109103913 learned_addr learned my addr 172.21.15.26:0/1109103913 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:46.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbefe59c640 1 -- 172.21.15.26:0/1109103913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf00107d20 msgr2=0x7fbf0019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbefe59c640 1 --2- 172.21.15.26:0/1109103913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf00107d20 0x7fbf0019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbefe59c640 1 -- 172.21.15.26:0/1109103913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf001053f0 msgr2=0x7fbf00198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbefe59c640 1 --2- 172.21.15.26:0/1109103913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf001053f0 0x7fbf00198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbefe59c640 1 -- 172.21.15.26:0/1109103913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0019ff50 con 0x7fbf00102ac0 2024-04-21T10:30:46.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbefe59c640 1 --2- 172.21.15.26:0/1109103913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf00102ac0 0x7fbf00198330 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbef0005cc0 tx=0x7fbef0034830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:46.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbeef7fe640 1 -- 172.21.15.26:0/1109103913 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0039040 con 0x7fbf00102ac0 2024-04-21T10:30:46.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbeef7fe640 1 -- 172.21.15.26:0/1109103913 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef0044070 con 0x7fbf00102ac0 2024-04-21T10:30:46.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbf04881640 1 -- 172.21.15.26:0/1109103913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf001a01e0 con 0x7fbf00102ac0 2024-04-21T10:30:46.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbeef7fe640 1 -- 172.21.15.26:0/1109103913 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef00042a0 con 0x7fbf00102ac0 2024-04-21T10:30:46.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.066+0000 7fbf04881640 1 -- 172.21.15.26:0/1109103913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf001a06a0 con 0x7fbf00102ac0 2024-04-21T10:30:46.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.067+0000 7fbf04881640 1 -- 172.21.15.26:0/1109103913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf000fdf00 con 0x7fbf00102ac0 2024-04-21T10:30:46.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.068+0000 7fbeef7fe640 1 -- 172.21.15.26:0/1109103913 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbef0003970 con 0x7fbf00102ac0 2024-04-21T10:30:46.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.069+0000 7fbeef7fe640 1 --2- 172.21.15.26:0/1109103913 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbee80a3bb0 0x7fbee80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:46.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.069+0000 7fbefdd9b640 1 --2- 172.21.15.26:0/1109103913 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbee80a3bb0 0x7fbee80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:46.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.069+0000 7fbeef7fe640 1 -- 172.21.15.26:0/1109103913 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fbef00f3510 con 0x7fbf00102ac0 2024-04-21T10:30:46.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.069+0000 7fbefdd9b640 1 --2- 172.21.15.26:0/1109103913 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbee80a3bb0 0x7fbee80a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf00199840 tx=0x7fbef4006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:46.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.073+0000 7fbeef7fe640 1 -- 172.21.15.26:0/1109103913 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbef00bfe80 con 0x7fbf00102ac0 2024-04-21T10:30:46.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.260+0000 7fbf04881640 1 -- 172.21.15.26:0/1109103913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf0002d5c0 con 0x7fbf00102ac0 2024-04-21T10:30:46.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.260+0000 7fbeef7fe640 1 -- 172.21.15.26:0/1109103913 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fbef00bf780 con 0x7fbf00102ac0 2024-04-21T10:30:46.260 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-21T10:30:46.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.262+0000 7fbf04881640 1 -- 172.21.15.26:0/1109103913 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbee80a3bb0 msgr2=0x7fbee80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.262+0000 7fbf04881640 1 --2- 172.21.15.26:0/1109103913 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbee80a3bb0 0x7fbee80a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf00199840 tx=0x7fbef4006c90 comp rx=0 tx=0).stop 2024-04-21T10:30:46.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.262+0000 7fbf04881640 1 -- 172.21.15.26:0/1109103913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf00102ac0 msgr2=0x7fbf00198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.262+0000 7fbf04881640 1 --2- 172.21.15.26:0/1109103913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf00102ac0 0x7fbf00198330 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbef0005cc0 tx=0x7fbef0034830 comp rx=0 tx=0).stop 2024-04-21T10:30:46.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.262+0000 7fbf04881640 1 -- 172.21.15.26:0/1109103913 shutdown_connections 2024-04-21T10:30:46.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.262+0000 7fbf04881640 1 --2- 172.21.15.26:0/1109103913 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fbee80a3bb0 0x7fbee80a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.262+0000 7fbf04881640 1 --2- 172.21.15.26:0/1109103913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf00107d20 0x7fbf0019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.262+0000 7fbf04881640 1 --2- 172.21.15.26:0/1109103913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf001053f0 0x7fbf00198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.262+0000 7fbf04881640 1 --2- 172.21.15.26:0/1109103913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf00102ac0 0x7fbf00198330 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.262+0000 7fbf04881640 1 -- 172.21.15.26:0/1109103913 >> 172.21.15.26:0/1109103913 conn(0x7fbf000fc410 msgr2=0x7fbf00103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:46.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.262+0000 7fbf04881640 1 -- 172.21.15.26:0/1109103913 shutdown_connections 2024-04-21T10:30:46.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.262+0000 7fbf04881640 1 -- 172.21.15.26:0/1109103913 wait complete. 2024-04-21T10:30:46.273 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T10:30:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.378+0000 7f653ac4a640 1 -- 172.21.15.26:0/3449141003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6534101fe0 msgr2=0x7f6534102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.378+0000 7f653ac4a640 1 --2- 172.21.15.26:0/3449141003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6534101fe0 0x7f6534102460 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6528009900 tx=0x7f652802bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.378+0000 7f653ac4a640 1 -- 172.21.15.26:0/3449141003 shutdown_connections 2024-04-21T10:30:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.378+0000 7f653ac4a640 1 --2- 172.21.15.26:0/3449141003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65341029a0 0x7f65341092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.378+0000 7f653ac4a640 1 --2- 172.21.15.26:0/3449141003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6534101fe0 0x7f6534102460 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.378+0000 7f653ac4a640 1 --2- 172.21.15.26:0/3449141003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6534100e90 0x7f65341012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.378+0000 7f653ac4a640 1 -- 172.21.15.26:0/3449141003 >> 172.21.15.26:0/3449141003 conn(0x7f65340fc400 msgr2=0x7f65340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.378+0000 7f653ac4a640 1 -- 172.21.15.26:0/3449141003 shutdown_connections 2024-04-21T10:30:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.378+0000 7f653ac4a640 1 -- 172.21.15.26:0/3449141003 wait complete. 2024-04-21T10:30:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f653ac4a640 1 Processor -- start 2024-04-21T10:30:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f653ac4a640 1 -- start start 2024-04-21T10:30:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f653ac4a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6534100e90 0x7f6534069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f653ac4a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6534101fe0 0x7f653419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f653ac4a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65341029a0 0x7f65341a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f653ac4a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f653410ca20 con 0x7f6534100e90 2024-04-21T10:30:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f653ac4a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f653410c8a0 con 0x7f65341029a0 2024-04-21T10:30:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f653ac4a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f653410cba0 con 0x7f6534101fe0 2024-04-21T10:30:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f65389e6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6534100e90 0x7f6534069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f65389e6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6534100e90 0x7f6534069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58146/0 (socket says 172.21.15.26:58146) 2024-04-21T10:30:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f65389e6640 1 -- 172.21.15.26:0/1194551800 learned_addr learned my addr 172.21.15.26:0/1194551800 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f6533fff640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6534101fe0 0x7f653419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.379+0000 7f65391e7640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65341029a0 0x7f65341a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.380+0000 7f65389e6640 1 -- 172.21.15.26:0/1194551800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6534101fe0 msgr2=0x7f653419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.380+0000 7f65389e6640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6534101fe0 0x7f653419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.380+0000 7f65389e6640 1 -- 172.21.15.26:0/1194551800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65341029a0 msgr2=0x7f65341a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.380+0000 7f65389e6640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65341029a0 0x7f65341a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.380+0000 7f65389e6640 1 -- 172.21.15.26:0/1194551800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65341a2050 con 0x7f6534100e90 2024-04-21T10:30:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.380+0000 7f65389e6640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6534100e90 0x7f6534069c50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f652400c890 tx=0x7f652400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.380+0000 7f6531ffb640 1 -- 172.21.15.26:0/1194551800 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524004220 con 0x7f6534100e90 2024-04-21T10:30:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.380+0000 7f6531ffb640 1 -- 172.21.15.26:0/1194551800 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65240043c0 con 0x7f6534100e90 2024-04-21T10:30:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.380+0000 7f653ac4a640 1 -- 172.21.15.26:0/1194551800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65341a22e0 con 0x7f6534100e90 2024-04-21T10:30:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.380+0000 7f6531ffb640 1 -- 172.21.15.26:0/1194551800 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524004a00 con 0x7f6534100e90 2024-04-21T10:30:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.380+0000 7f653ac4a640 1 -- 172.21.15.26:0/1194551800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65341a2800 con 0x7f6534100e90 2024-04-21T10:30:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.381+0000 7f653ac4a640 1 -- 172.21.15.26:0/1194551800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6534068d70 con 0x7f6534100e90 2024-04-21T10:30:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.382+0000 7f6531ffb640 1 -- 172.21.15.26:0/1194551800 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f65240079b0 con 0x7f6534100e90 2024-04-21T10:30:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.383+0000 7f6531ffb640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f651c0a3be0 0x7f651c0a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.383+0000 7f6533fff640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f651c0a3be0 0x7f651c0a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.383+0000 7f6531ffb640 1 -- 172.21.15.26:0/1194551800 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f65240d0160 con 0x7f6534100e90 2024-04-21T10:30:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.383+0000 7f6533fff640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f651c0a3be0 0x7f651c0a60a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f652802c240 tx=0x7f6528000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:46.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.387+0000 7f6531ffb640 1 -- 172.21.15.26:0/1194551800 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6524014030 con 0x7f6534100e90 2024-04-21T10:30:46.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.573+0000 7f653ac4a640 1 -- 172.21.15.26:0/1194551800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6534062b50 con 0x7f6534100e90 2024-04-21T10:30:46.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.573+0000 7f6531ffb640 1 -- 172.21.15.26:0/1194551800 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f652409cad0 con 0x7f6534100e90 2024-04-21T10:30:46.573 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-21T10:30:46.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.575+0000 7f653ac4a640 1 -- 172.21.15.26:0/1194551800 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f651c0a3be0 msgr2=0x7f651c0a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.575+0000 7f653ac4a640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f651c0a3be0 0x7f651c0a60a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f652802c240 tx=0x7f6528000bc0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.575+0000 7f653ac4a640 1 -- 172.21.15.26:0/1194551800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6534100e90 msgr2=0x7f6534069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.575+0000 7f653ac4a640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6534100e90 0x7f6534069c50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f652400c890 tx=0x7f652400cd50 comp rx=0 tx=0).stop 2024-04-21T10:30:46.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.575+0000 7f653ac4a640 1 -- 172.21.15.26:0/1194551800 shutdown_connections 2024-04-21T10:30:46.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.575+0000 7f653ac4a640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f651c0a3be0 0x7f651c0a60a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.575+0000 7f653ac4a640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65341029a0 0x7f65341a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.575+0000 7f653ac4a640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6534101fe0 0x7f653419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.575+0000 7f653ac4a640 1 --2- 172.21.15.26:0/1194551800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6534100e90 0x7f6534069c50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.575+0000 7f653ac4a640 1 -- 172.21.15.26:0/1194551800 >> 172.21.15.26:0/1194551800 conn(0x7f65340fc400 msgr2=0x7f6534105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:46.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.575+0000 7f653ac4a640 1 -- 172.21.15.26:0/1194551800 shutdown_connections 2024-04-21T10:30:46.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.575+0000 7f653ac4a640 1 -- 172.21.15.26:0/1194551800 wait complete. 2024-04-21T10:30:46.585 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:46 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1080159068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:46.585 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:46 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1109103913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:30:46.586 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T10:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.691+0000 7f1c233f4640 1 -- 172.21.15.26:0/14631293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c1031a0 msgr2=0x7f1c1c103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.691+0000 7f1c233f4640 1 --2- 172.21.15.26:0/14631293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c103620 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009900 tx=0x7f1c0c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.691+0000 7f1c233f4640 1 -- 172.21.15.26:0/14631293 shutdown_connections 2024-04-21T10:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.691+0000 7f1c233f4640 1 --2- 172.21.15.26:0/14631293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c1c103b60 0x7f1c1c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.691+0000 7f1c233f4640 1 --2- 172.21.15.26:0/14631293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c103620 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.691+0000 7f1c233f4640 1 --2- 172.21.15.26:0/14631293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c1c101fb0 0x7f1c1c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.691+0000 7f1c233f4640 1 -- 172.21.15.26:0/14631293 >> 172.21.15.26:0/14631293 conn(0x7f1c1c0fd520 msgr2=0x7f1c1c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.691+0000 7f1c233f4640 1 -- 172.21.15.26:0/14631293 shutdown_connections 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.691+0000 7f1c233f4640 1 -- 172.21.15.26:0/14631293 wait complete. 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c233f4640 1 Processor -- start 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c233f4640 1 -- start start 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c233f4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c101fb0 0x7f1c1c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c233f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c233f4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c1c103b60 0x7f1c1c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c233f4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10daf0 con 0x7f1c1c101fb0 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c233f4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10d970 con 0x7f1c1c103b60 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c233f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10dc70 con 0x7f1c1c1031a0 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c21190640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c101fb0 0x7f1c1c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c21190640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c101fb0 0x7f1c1c1981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58160/0 (socket says 172.21.15.26:58160) 2024-04-21T10:30:46.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c21190640 1 -- 172.21.15.26:0/2755050644 learned_addr learned my addr 172.21.15.26:0/2755050644 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c2098f640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.692+0000 7f1c21991640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c1c103b60 0x7f1c1c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.693+0000 7f1c21190640 1 -- 172.21.15.26:0/2755050644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c1c1031a0 msgr2=0x7f1c1c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.693+0000 7f1c21190640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.693+0000 7f1c21190640 1 -- 172.21.15.26:0/2755050644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c1c103b60 msgr2=0x7f1c1c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.693+0000 7f1c21190640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c1c103b60 0x7f1c1c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.693+0000 7f1c21190640 1 -- 172.21.15.26:0/2755050644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c19fdf0 con 0x7f1c1c101fb0 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.693+0000 7f1c2098f640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.693+0000 7f1c21190640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c101fb0 0x7f1c1c1981d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800e9c0 tx=0x7f1c1800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.693+0000 7f1c127fc640 1 -- 172.21.15.26:0/2755050644 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800cd80 con 0x7f1c1c101fb0 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.693+0000 7f1c127fc640 1 -- 172.21.15.26:0/2755050644 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c180035f0 con 0x7f1c1c101fb0 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.693+0000 7f1c233f4640 1 -- 172.21.15.26:0/2755050644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c1a00e0 con 0x7f1c1c101fb0 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.693+0000 7f1c127fc640 1 -- 172.21.15.26:0/2755050644 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18010690 con 0x7f1c1c101fb0 2024-04-21T10:30:46.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.693+0000 7f1c233f4640 1 -- 172.21.15.26:0/2755050644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c1a0630 con 0x7f1c1c101fb0 2024-04-21T10:30:46.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.694+0000 7f1c233f4640 1 -- 172.21.15.26:0/2755050644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1c109350 con 0x7f1c1c101fb0 2024-04-21T10:30:46.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.695+0000 7f1c127fc640 1 -- 172.21.15.26:0/2755050644 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1c180037a0 con 0x7f1c1c101fb0 2024-04-21T10:30:46.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.696+0000 7f1c127fc640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f1c080a3bb0 0x7f1c080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:46.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.696+0000 7f1c2098f640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f1c080a3bb0 0x7f1c080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:46.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.696+0000 7f1c127fc640 1 -- 172.21.15.26:0/2755050644 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f1c180d08e0 con 0x7f1c1c101fb0 2024-04-21T10:30:46.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.696+0000 7f1c2098f640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f1c080a3bb0 0x7f1c080a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c1996e0 tx=0x7f1c0c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:46.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:46 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1080159068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:46.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:46 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1109103913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:30:46.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.700+0000 7f1c127fc640 1 -- 172.21.15.26:0/2755050644 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c1809d2b0 con 0x7f1c1c101fb0 2024-04-21T10:30:46.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:46 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1080159068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:46.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:46 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1109103913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:30:46.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.890+0000 7f1c233f4640 1 -- 172.21.15.26:0/2755050644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c1c062b50 con 0x7f1c1c101fb0 2024-04-21T10:30:46.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.890+0000 7f1c127fc640 1 -- 172.21.15.26:0/2755050644 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f1c1809cbb0 con 0x7f1c1c101fb0 2024-04-21T10:30:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-21T10:30:46.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.892+0000 7f1c233f4640 1 -- 172.21.15.26:0/2755050644 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f1c080a3bb0 msgr2=0x7f1c080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.892+0000 7f1c233f4640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f1c080a3bb0 0x7f1c080a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c1996e0 tx=0x7f1c0c02c7e0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.892+0000 7f1c233f4640 1 -- 172.21.15.26:0/2755050644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c101fb0 msgr2=0x7f1c1c1981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:46.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.892+0000 7f1c233f4640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c101fb0 0x7f1c1c1981d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800e9c0 tx=0x7f1c1800ee80 comp rx=0 tx=0).stop 2024-04-21T10:30:46.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.892+0000 7f1c233f4640 1 -- 172.21.15.26:0/2755050644 shutdown_connections 2024-04-21T10:30:46.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.892+0000 7f1c233f4640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f1c080a3bb0 0x7f1c080a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.892+0000 7f1c233f4640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c1c103b60 0x7f1c1c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.892+0000 7f1c233f4640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c1c1031a0 0x7f1c1c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.892+0000 7f1c233f4640 1 --2- 172.21.15.26:0/2755050644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c101fb0 0x7f1c1c1981d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:46.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.892+0000 7f1c233f4640 1 -- 172.21.15.26:0/2755050644 >> 172.21.15.26:0/2755050644 conn(0x7f1c1c0fd520 msgr2=0x7f1c1c106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:46.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.892+0000 7f1c233f4640 1 -- 172.21.15.26:0/2755050644 shutdown_connections 2024-04-21T10:30:46.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:46.892+0000 7f1c233f4640 1 -- 172.21.15.26:0/2755050644 wait complete. 2024-04-21T10:30:46.903 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:30:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.005+0000 7effc9811640 1 -- 172.21.15.26:0/3537899946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc4100260 msgr2=0x7effc4100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.005+0000 7effc9811640 1 --2- 172.21.15.26:0/3537899946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc4100260 0x7effc4100680 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7effb4009930 tx=0x7effb402bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.005+0000 7effc9811640 1 -- 172.21.15.26:0/3537899946 shutdown_connections 2024-04-21T10:30:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.005+0000 7effc9811640 1 --2- 172.21.15.26:0/3537899946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effc4103bf0 0x7effc410a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.005+0000 7effc9811640 1 --2- 172.21.15.26:0/3537899946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc4103230 0x7effc41036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.005+0000 7effc9811640 1 --2- 172.21.15.26:0/3537899946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc4100260 0x7effc4100680 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.005+0000 7effc9811640 1 -- 172.21.15.26:0/3537899946 >> 172.21.15.26:0/3537899946 conn(0x7effc40fbb90 msgr2=0x7effc40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.005+0000 7effc9811640 1 -- 172.21.15.26:0/3537899946 shutdown_connections 2024-04-21T10:30:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.005+0000 7effc9811640 1 -- 172.21.15.26:0/3537899946 wait complete. 2024-04-21T10:30:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effc9811640 1 Processor -- start 2024-04-21T10:30:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effc9811640 1 -- start start 2024-04-21T10:30:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effc9811640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effc4100260 0x7effc41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effc9811640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc4103230 0x7effc4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effc9811640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc4103bf0 0x7effc419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effc9811640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7effc410db00 con 0x7effc4103230 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effc9811640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7effc410d980 con 0x7effc4100260 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effc9811640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7effc410dc80 con 0x7effc4103bf0 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effbbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc4103230 0x7effc4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effbbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc4103230 0x7effc4198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58180/0 (socket says 172.21.15.26:58180) 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effbbfff640 1 -- 172.21.15.26:0/1221272605 learned_addr learned my addr 172.21.15.26:0/1221272605 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effbbfff640 1 -- 172.21.15.26:0/1221272605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc4103bf0 msgr2=0x7effc419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effbbfff640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc4103bf0 0x7effc419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effbbfff640 1 -- 172.21.15.26:0/1221272605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effc4100260 msgr2=0x7effc41981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effc2ffd640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effc4100260 0x7effc41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effbbfff640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effc4100260 0x7effc41981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.006+0000 7effbbfff640 1 -- 172.21.15.26:0/1221272605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc419fde0 con 0x7effc4103230 2024-04-21T10:30:47.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.007+0000 7effbbfff640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc4103230 0x7effc4198700 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7effac00d950 tx=0x7effac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:47.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.007+0000 7effc880f640 1 -- 172.21.15.26:0/1221272605 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac017030 con 0x7effc4103230 2024-04-21T10:30:47.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.007+0000 7effc9811640 1 -- 172.21.15.26:0/1221272605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc41a00d0 con 0x7effc4103230 2024-04-21T10:30:47.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.007+0000 7effc880f640 1 -- 172.21.15.26:0/1221272605 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effac004020 con 0x7effc4103230 2024-04-21T10:30:47.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.007+0000 7effc880f640 1 -- 172.21.15.26:0/1221272605 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0047a0 con 0x7effc4103230 2024-04-21T10:30:47.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.007+0000 7effc9811640 1 -- 172.21.15.26:0/1221272605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc41a0620 con 0x7effc4103230 2024-04-21T10:30:47.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.008+0000 7effc9811640 1 -- 172.21.15.26:0/1221272605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc4109430 con 0x7effc4103230 2024-04-21T10:30:47.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.009+0000 7effc880f640 1 -- 172.21.15.26:0/1221272605 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7effac009a30 con 0x7effc4103230 2024-04-21T10:30:47.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.009+0000 7effc880f640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7effb00a3bb0 0x7effb00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:47.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.010+0000 7effc2ffd640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7effb00a3bb0 0x7effb00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:47.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.010+0000 7effc880f640 1 -- 172.21.15.26:0/1221272605 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7effac0cffe0 con 0x7effc4103230 2024-04-21T10:30:47.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.010+0000 7effc2ffd640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7effb00a3bb0 0x7effb00a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7effb402c840 tx=0x7effb4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:47.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.014+0000 7effc880f640 1 -- 172.21.15.26:0/1221272605 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effac09c990 con 0x7effc4103230 2024-04-21T10:30:47.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.239+0000 7effc9811640 1 -- 172.21.15.26:0/1221272605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effc4062bf0 con 0x7effc4103230 2024-04-21T10:30:47.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.240+0000 7effc880f640 1 -- 172.21.15.26:0/1221272605 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11884 (secure 0 0 0) 0x7effac09c290 con 0x7effc4103230 2024-04-21T10:30:47.240 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:47.240 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:29:05.806617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24478,24488,24496]},"id":1}]} 2024-04-21T10:30:47.240 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 20 2024-04-21T10:30:47.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.242+0000 7effc9811640 1 -- 172.21.15.26:0/1221272605 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7effb00a3bb0 msgr2=0x7effb00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.242+0000 7effc9811640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7effb00a3bb0 0x7effb00a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7effb402c840 tx=0x7effb4000bc0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.242+0000 7effc9811640 1 -- 172.21.15.26:0/1221272605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc4103230 msgr2=0x7effc4198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.242+0000 7effc9811640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc4103230 0x7effc4198700 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7effac00d950 tx=0x7effac00de10 comp rx=0 tx=0).stop 2024-04-21T10:30:47.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.242+0000 7effc9811640 1 -- 172.21.15.26:0/1221272605 shutdown_connections 2024-04-21T10:30:47.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.242+0000 7effc9811640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7effb00a3bb0 0x7effb00a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.242+0000 7effc9811640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effc4103bf0 0x7effc419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.242+0000 7effc9811640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc4103230 0x7effc4198700 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.242+0000 7effc9811640 1 --2- 172.21.15.26:0/1221272605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effc4100260 0x7effc41981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.242+0000 7effc9811640 1 -- 172.21.15.26:0/1221272605 >> 172.21.15.26:0/1221272605 conn(0x7effc40fbb90 msgr2=0x7effc4106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:47.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.242+0000 7effc9811640 1 -- 172.21.15.26:0/1221272605 shutdown_connections 2024-04-21T10:30:47.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.242+0000 7effc9811640 1 -- 172.21.15.26:0/1221272605 wait complete. 2024-04-21T10:30:47.254 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:30:47.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.355+0000 7fd91cdce640 1 -- 172.21.15.26:0/1694860079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd918102980 msgr2=0x7fd9181092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.355+0000 7fd91cdce640 1 --2- 172.21.15.26:0/1694860079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd918102980 0x7fd9181092a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd910009ca0 tx=0x7fd91002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.355+0000 7fd91cdce640 1 -- 172.21.15.26:0/1694860079 shutdown_connections 2024-04-21T10:30:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.355+0000 7fd91cdce640 1 --2- 172.21.15.26:0/1694860079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd918102980 0x7fd9181092a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.355+0000 7fd91cdce640 1 --2- 172.21.15.26:0/1694860079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd918101fc0 0x7fd918102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.355+0000 7fd91cdce640 1 --2- 172.21.15.26:0/1694860079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd918100e70 0x7fd918101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.355+0000 7fd91cdce640 1 -- 172.21.15.26:0/1694860079 >> 172.21.15.26:0/1694860079 conn(0x7fd9180fc3e0 msgr2=0x7fd9180fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.356+0000 7fd91cdce640 1 -- 172.21.15.26:0/1694860079 shutdown_connections 2024-04-21T10:30:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.356+0000 7fd91cdce640 1 -- 172.21.15.26:0/1694860079 wait complete. 2024-04-21T10:30:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.356+0000 7fd91cdce640 1 Processor -- start 2024-04-21T10:30:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.356+0000 7fd91cdce640 1 -- start start 2024-04-21T10:30:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.356+0000 7fd91cdce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd918100e70 0x7fd918198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.356+0000 7fd91cdce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd918101fc0 0x7fd9181986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.356+0000 7fd91cdce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd918102980 0x7fd91819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.356+0000 7fd91cdce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd91810c9c0 con 0x7fd918100e70 2024-04-21T10:30:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91cdce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd91810c840 con 0x7fd918102980 2024-04-21T10:30:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91cdce640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd91810cb40 con 0x7fd918101fc0 2024-04-21T10:30:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd918100e70 0x7fd918198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd918100e70 0x7fd918198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58192/0 (socket says 172.21.15.26:58192) 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91659c640 1 -- 172.21.15.26:0/3343812054 learned_addr learned my addr 172.21.15.26:0/3343812054 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd915d9b640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd918101fc0 0x7fd9181986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91659c640 1 -- 172.21.15.26:0/3343812054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd918101fc0 msgr2=0x7fd9181986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91659c640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd918101fc0 0x7fd9181986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91659c640 1 -- 172.21.15.26:0/3343812054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd918102980 msgr2=0x7fd91819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91659c640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd918102980 0x7fd91819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91659c640 1 -- 172.21.15.26:0/3343812054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91819fd80 con 0x7fd918100e70 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91659c640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd918100e70 0x7fd918198160 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd90800d770 tx=0x7fd90800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd9077fe640 1 -- 172.21.15.26:0/3343812054 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908017070 con 0x7fd918100e70 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd9077fe640 1 -- 172.21.15.26:0/3343812054 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9080054d0 con 0x7fd918100e70 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91cdce640 1 -- 172.21.15.26:0/3343812054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9181a0070 con 0x7fd918100e70 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd9077fe640 1 -- 172.21.15.26:0/3343812054 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908004c30 con 0x7fd918100e70 2024-04-21T10:30:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.357+0000 7fd91cdce640 1 -- 172.21.15.26:0/3343812054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9181a05c0 con 0x7fd918100e70 2024-04-21T10:30:47.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.358+0000 7fd91cdce640 1 -- 172.21.15.26:0/3343812054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9181a0820 con 0x7fd918100e70 2024-04-21T10:30:47.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.360+0000 7fd9077fe640 1 -- 172.21.15.26:0/3343812054 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd908004dd0 con 0x7fd918100e70 2024-04-21T10:30:47.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.360+0000 7fd9077fe640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd9000a3c30 0x7fd9000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:47.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.360+0000 7fd9077fe640 1 -- 172.21.15.26:0/3343812054 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fd9080d0330 con 0x7fd918100e70 2024-04-21T10:30:47.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.360+0000 7fd915d9b640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd9000a3c30 0x7fd9000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:47.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.361+0000 7fd915d9b640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd9000a3c30 0x7fd9000a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd918199670 tx=0x7fd90c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:47.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.364+0000 7fd9077fe640 1 -- 172.21.15.26:0/3343812054 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd90809cd80 con 0x7fd918100e70 2024-04-21T10:30:47.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.553+0000 7fd91cdce640 1 -- 172.21.15.26:0/3343812054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9181a0b10 con 0x7fd918100e70 2024-04-21T10:30:47.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.554+0000 7fd9077fe640 1 -- 172.21.15.26:0/3343812054 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22705 (secure 0 0 0) 0x7fd90809c680 con 0x7fd918100e70 2024-04-21T10:30:47.554 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:47.555 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:30:03.297428+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:30:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.556+0000 7fd91cdce640 1 -- 172.21.15.26:0/3343812054 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd9000a3c30 msgr2=0x7fd9000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.556+0000 7fd91cdce640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd9000a3c30 0x7fd9000a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd918199670 tx=0x7fd90c00b5b0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.556+0000 7fd91cdce640 1 -- 172.21.15.26:0/3343812054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd918100e70 msgr2=0x7fd918198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.556+0000 7fd91cdce640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd918100e70 0x7fd918198160 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd90800d770 tx=0x7fd90800dc30 comp rx=0 tx=0).stop 2024-04-21T10:30:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.556+0000 7fd91cdce640 1 -- 172.21.15.26:0/3343812054 shutdown_connections 2024-04-21T10:30:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.556+0000 7fd91cdce640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fd9000a3c30 0x7fd9000a60f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.556+0000 7fd91cdce640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd918102980 0x7fd91819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.556+0000 7fd91cdce640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd918101fc0 0x7fd9181986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.556+0000 7fd91cdce640 1 --2- 172.21.15.26:0/3343812054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd918100e70 0x7fd918198160 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.556+0000 7fd91cdce640 1 -- 172.21.15.26:0/3343812054 >> 172.21.15.26:0/3343812054 conn(0x7fd9180fc3e0 msgr2=0x7fd9180fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.556+0000 7fd91cdce640 1 -- 172.21.15.26:0/3343812054 shutdown_connections 2024-04-21T10:30:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.556+0000 7fd91cdce640 1 -- 172.21.15.26:0/3343812054 wait complete. 2024-04-21T10:30:47.566 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:47 smithi026 ceph-mon[27965]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:30:47.566 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:47 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1194551800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:30:47.567 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:47 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2755050644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:30:47.567 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:47 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1221272605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:47.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T10:24:28.176891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T10:28:59.417687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T10:29:00.234808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T10:30:47.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T10:30:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 -- 172.21.15.26:0/2916025166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f635c10c980 msgr2=0x7f635c10cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 --2- 172.21.15.26:0/2916025166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f635c10c980 0x7f635c10cda0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6358009900 tx=0x7f635802bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 -- 172.21.15.26:0/2916025166 shutdown_connections 2024-04-21T10:30:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 --2- 172.21.15.26:0/2916025166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f635c10e530 0x7f635c114dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 --2- 172.21.15.26:0/2916025166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c10db70 0x7f635c10dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 --2- 172.21.15.26:0/2916025166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f635c10c980 0x7f635c10cda0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 -- 172.21.15.26:0/2916025166 >> 172.21.15.26:0/2916025166 conn(0x7f635c06c9e0 msgr2=0x7f635c06ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 -- 172.21.15.26:0/2916025166 shutdown_connections 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 -- 172.21.15.26:0/2916025166 wait complete. 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 Processor -- start 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 -- start start 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f635c10c980 0x7f635c1a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f635c10db70 0x7f635c1a3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c10e530 0x7f635c1aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f635c1184b0 con 0x7f635c10c980 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f635c118330 con 0x7f635c10e530 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.675+0000 7f6363862640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f635c118630 con 0x7f635c10db70 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f63615fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f635c10c980 0x7f635c1a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f63615fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f635c10c980 0x7f635c1a2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58212/0 (socket says 172.21.15.26:58212) 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f63615fe640 1 -- 172.21.15.26:0/3281788869 learned_addr learned my addr 172.21.15.26:0/3281788869 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f63615fe640 1 -- 172.21.15.26:0/3281788869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f635c10db70 msgr2=0x7f635c1a3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f63615fe640 1 --2- 172.21.15.26:0/3281788869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f635c10db70 0x7f635c1a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f63615fe640 1 -- 172.21.15.26:0/3281788869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c10e530 msgr2=0x7f635c1aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f63615fe640 1 --2- 172.21.15.26:0/3281788869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c10e530 0x7f635c1aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f63615fe640 1 -- 172.21.15.26:0/3281788869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c1aa840 con 0x7f635c10c980 2024-04-21T10:30:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f63615fe640 1 --2- 172.21.15.26:0/3281788869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f635c10c980 0x7f635c1a2c30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6358000c00 tx=0x7f6358004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f63527fc640 1 -- 172.21.15.26:0/3281788869 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358043020 con 0x7f635c10c980 2024-04-21T10:30:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f63527fc640 1 -- 172.21.15.26:0/3281788869 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6358004aa0 con 0x7f635c10c980 2024-04-21T10:30:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f6363862640 1 -- 172.21.15.26:0/3281788869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635c1aaad0 con 0x7f635c10c980 2024-04-21T10:30:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f63527fc640 1 -- 172.21.15.26:0/3281788869 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358003d00 con 0x7f635c10c980 2024-04-21T10:30:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.676+0000 7f6363862640 1 -- 172.21.15.26:0/3281788869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635c1ab020 con 0x7f635c10c980 2024-04-21T10:30:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.677+0000 7f6363862640 1 -- 172.21.15.26:0/3281788869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635c113e10 con 0x7f635c10c980 2024-04-21T10:30:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.678+0000 7f63527fc640 1 -- 172.21.15.26:0/3281788869 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6358046020 con 0x7f635c10c980 2024-04-21T10:30:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.679+0000 7f63527fc640 1 --2- 172.21.15.26:0/3281788869 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f63480a3bb0 0x7f63480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.679+0000 7f6360dfd640 1 --2- 172.21.15.26:0/3281788869 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f63480a3bb0 0x7f63480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.679+0000 7f63527fc640 1 -- 172.21.15.26:0/3281788869 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f63580f2f90 con 0x7f635c10c980 2024-04-21T10:30:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.679+0000 7f6360dfd640 1 --2- 172.21.15.26:0/3281788869 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f63480a3bb0 0x7f63480a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f634c005fd0 tx=0x7f634c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.683+0000 7f63527fc640 1 -- 172.21.15.26:0/3281788869 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f635804d020 con 0x7f635c10c980 2024-04-21T10:30:47.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:47 smithi169 ceph-mon[31469]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:30:47.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:47 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1194551800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:30:47.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:47 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2755050644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:30:47.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:47 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1221272605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:47.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:47 smithi190 ceph-mon[31423]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:30:47.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:47 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1194551800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:30:47.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:47 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2755050644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:30:47.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:47 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1221272605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:47.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.867+0000 7f6363862640 1 -- 172.21.15.26:0/3281788869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f635c02d830 con 0x7f635c10c980 2024-04-21T10:30:47.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.868+0000 7f63527fc640 1 -- 172.21.15.26:0/3281788869 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f63580bf960 con 0x7f635c10c980 2024-04-21T10:30:47.868 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-21T10:30:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.870+0000 7f6363862640 1 -- 172.21.15.26:0/3281788869 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f63480a3bb0 msgr2=0x7f63480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.870+0000 7f6363862640 1 --2- 172.21.15.26:0/3281788869 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f63480a3bb0 0x7f63480a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f634c005fd0 tx=0x7f634c005850 comp rx=0 tx=0).stop 2024-04-21T10:30:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.870+0000 7f6363862640 1 -- 172.21.15.26:0/3281788869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f635c10c980 msgr2=0x7f635c1a2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.870+0000 7f6363862640 1 --2- 172.21.15.26:0/3281788869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f635c10c980 0x7f635c1a2c30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6358000c00 tx=0x7f6358004270 comp rx=0 tx=0).stop 2024-04-21T10:30:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.870+0000 7f6363862640 1 -- 172.21.15.26:0/3281788869 shutdown_connections 2024-04-21T10:30:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.870+0000 7f6363862640 1 --2- 172.21.15.26:0/3281788869 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f63480a3bb0 0x7f63480a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.870+0000 7f6363862640 1 --2- 172.21.15.26:0/3281788869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c10e530 0x7f635c1aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.870+0000 7f6363862640 1 --2- 172.21.15.26:0/3281788869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f635c10db70 0x7f635c1a3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.870+0000 7f6363862640 1 --2- 172.21.15.26:0/3281788869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f635c10c980 0x7f635c1a2c30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.870+0000 7f6363862640 1 -- 172.21.15.26:0/3281788869 >> 172.21.15.26:0/3281788869 conn(0x7f635c06c9e0 msgr2=0x7f635c110fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.870+0000 7f6363862640 1 -- 172.21.15.26:0/3281788869 shutdown_connections 2024-04-21T10:30:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:47.870+0000 7f6363862640 1 -- 172.21.15.26:0/3281788869 wait complete. 2024-04-21T10:30:48.101 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T10:30:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.207+0000 7f7e401b2640 1 -- 172.21.15.26:0/3305237544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e38107cf0 msgr2=0x7f7e3810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.207+0000 7f7e401b2640 1 --2- 172.21.15.26:0/3305237544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e38107cf0 0x7f7e3810a0e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009ca0 tx=0x7f7e3002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 -- 172.21.15.26:0/3305237544 shutdown_connections 2024-04-21T10:30:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 --2- 172.21.15.26:0/3305237544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e38107cf0 0x7f7e3810a0e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 --2- 172.21.15.26:0/3305237544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e381053c0 0x7f7e381077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 --2- 172.21.15.26:0/3305237544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38102a90 0x7f7e38104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 -- 172.21.15.26:0/3305237544 >> 172.21.15.26:0/3305237544 conn(0x7f7e380fc440 msgr2=0x7f7e380fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 -- 172.21.15.26:0/3305237544 shutdown_connections 2024-04-21T10:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 -- 172.21.15.26:0/3305237544 wait complete. 2024-04-21T10:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 Processor -- start 2024-04-21T10:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 -- start start 2024-04-21T10:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38102a90 0x7f7e38198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e381053c0 0x7f7e381986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e38107cf0 0x7f7e3819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3810db50 con 0x7f7e38107cf0 2024-04-21T10:30:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3810d9d0 con 0x7f7e38102a90 2024-04-21T10:30:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.208+0000 7f7e401b2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3810dcd0 con 0x7f7e381053c0 2024-04-21T10:30:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e3e74f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e38107cf0 0x7f7e3819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e3e74f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e38107cf0 0x7f7e3819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58228/0 (socket says 172.21.15.26:58228) 2024-04-21T10:30:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e3e74f640 1 -- 172.21.15.26:0/4111693026 learned_addr learned my addr 172.21.15.26:0/4111693026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e3d74d640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e381053c0 0x7f7e381986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e3df4e640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38102a90 0x7f7e38198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e3e74f640 1 -- 172.21.15.26:0/4111693026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e381053c0 msgr2=0x7f7e381986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e3e74f640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e381053c0 0x7f7e381986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e3e74f640 1 -- 172.21.15.26:0/4111693026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38102a90 msgr2=0x7f7e38198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e3e74f640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38102a90 0x7f7e38198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e3e74f640 1 -- 172.21.15.26:0/4111693026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3819fdb0 con 0x7f7e38107cf0 2024-04-21T10:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e3d74d640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e381053c0 0x7f7e381986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e3e74f640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e38107cf0 0x7f7e3819f740 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7e3002c700 tx=0x7f7e30035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e2effd640 1 -- 172.21.15.26:0/4111693026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30039040 con 0x7f7e38107cf0 2024-04-21T10:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e2effd640 1 -- 172.21.15.26:0/4111693026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e30044070 con 0x7f7e38107cf0 2024-04-21T10:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e2effd640 1 -- 172.21.15.26:0/4111693026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30004440 con 0x7f7e38107cf0 2024-04-21T10:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e401b2640 1 -- 172.21.15.26:0/4111693026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e381a0040 con 0x7f7e38107cf0 2024-04-21T10:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.209+0000 7f7e401b2640 1 -- 172.21.15.26:0/4111693026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e381a0580 con 0x7f7e38107cf0 2024-04-21T10:30:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.210+0000 7f7e401b2640 1 -- 172.21.15.26:0/4111693026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e380fdf30 con 0x7f7e38107cf0 2024-04-21T10:30:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.211+0000 7f7e2effd640 1 -- 172.21.15.26:0/4111693026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7e300045e0 con 0x7f7e38107cf0 2024-04-21T10:30:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.212+0000 7f7e2effd640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7e240a3c30 0x7f7e240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.212+0000 7f7e3df4e640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7e240a3c30 0x7f7e240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.212+0000 7f7e2effd640 1 -- 172.21.15.26:0/4111693026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f7e300f2ba0 con 0x7f7e38107cf0 2024-04-21T10:30:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.212+0000 7f7e3df4e640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7e240a3c30 0x7f7e240a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7e34005e10 tx=0x7f7e34005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.217+0000 7f7e2effd640 1 -- 172.21.15.26:0/4111693026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e300bf510 con 0x7f7e38107cf0 2024-04-21T10:30:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.402+0000 7f7e401b2640 1 -- 172.21.15.26:0/4111693026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7e38062b50 con 0x7f7e38107cf0 2024-04-21T10:30:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.403+0000 7f7e2effd640 1 -- 172.21.15.26:0/4111693026 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f7e300bee10 con 0x7f7e38107cf0 2024-04-21T10:30:48.403 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-21T10:30:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.405+0000 7f7e401b2640 1 -- 172.21.15.26:0/4111693026 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7e240a3c30 msgr2=0x7f7e240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.405+0000 7f7e401b2640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7e240a3c30 0x7f7e240a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7e34005e10 tx=0x7f7e34005da0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.405+0000 7f7e401b2640 1 -- 172.21.15.26:0/4111693026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e38107cf0 msgr2=0x7f7e3819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.405+0000 7f7e401b2640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e38107cf0 0x7f7e3819f740 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7e3002c700 tx=0x7f7e30035d60 comp rx=0 tx=0).stop 2024-04-21T10:30:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.405+0000 7f7e401b2640 1 -- 172.21.15.26:0/4111693026 shutdown_connections 2024-04-21T10:30:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.405+0000 7f7e401b2640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f7e240a3c30 0x7f7e240a60f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.405+0000 7f7e401b2640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e38107cf0 0x7f7e3819f740 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.405+0000 7f7e401b2640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e381053c0 0x7f7e381986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.405+0000 7f7e401b2640 1 --2- 172.21.15.26:0/4111693026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38102a90 0x7f7e38198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.405+0000 7f7e401b2640 1 -- 172.21.15.26:0/4111693026 >> 172.21.15.26:0/4111693026 conn(0x7f7e380fc440 msgr2=0x7f7e381033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.405+0000 7f7e401b2640 1 -- 172.21.15.26:0/4111693026 shutdown_connections 2024-04-21T10:30:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.405+0000 7f7e401b2640 1 -- 172.21.15.26:0/4111693026 wait complete. 2024-04-21T10:30:48.416 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T10:30:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.523+0000 7f532b631640 1 -- 172.21.15.26:0/1532137385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5324103b60 msgr2=0x7f532410a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.523+0000 7f532b631640 1 --2- 172.21.15.26:0/1532137385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5324103b60 0x7f532410a3f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f531c009ca0 tx=0x7f531c02c0f0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.523+0000 7f532b631640 1 -- 172.21.15.26:0/1532137385 shutdown_connections 2024-04-21T10:30:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.523+0000 7f532b631640 1 --2- 172.21.15.26:0/1532137385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5324103b60 0x7f532410a3f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.523+0000 7f532b631640 1 --2- 172.21.15.26:0/1532137385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53241031a0 0x7f5324103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.523+0000 7f532b631640 1 --2- 172.21.15.26:0/1532137385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5324101fb0 0x7f53241023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.523+0000 7f532b631640 1 -- 172.21.15.26:0/1532137385 >> 172.21.15.26:0/1532137385 conn(0x7f53240fd520 msgr2=0x7f53240ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.523+0000 7f532b631640 1 -- 172.21.15.26:0/1532137385 shutdown_connections 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.523+0000 7f532b631640 1 -- 172.21.15.26:0/1532137385 wait complete. 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f532b631640 1 Processor -- start 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f532b631640 1 -- start start 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f532b631640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5324101fb0 0x7f53241981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f532b631640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53241031a0 0x7f53241986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f532b631640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5324103b60 0x7f532419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f532b631640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f532410daf0 con 0x7f5324103b60 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f532b631640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f532410d970 con 0x7f53241031a0 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f532b631640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f532410dc70 con 0x7f5324101fb0 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f5329bce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5324103b60 0x7f532419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f5329bce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5324103b60 0x7f532419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58246/0 (socket says 172.21.15.26:58246) 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f5329bce640 1 -- 172.21.15.26:0/1038552722 learned_addr learned my addr 172.21.15.26:0/1038552722 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f53293cd640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5324101fb0 0x7f53241981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f5328bcc640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53241031a0 0x7f53241986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f5329bce640 1 -- 172.21.15.26:0/1038552722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5324101fb0 msgr2=0x7f53241981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f5329bce640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5324101fb0 0x7f53241981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f5329bce640 1 -- 172.21.15.26:0/1038552722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53241031a0 msgr2=0x7f53241986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f5329bce640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53241031a0 0x7f53241986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.524+0000 7f5329bce640 1 -- 172.21.15.26:0/1038552722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532419fdc0 con 0x7f5324103b60 2024-04-21T10:30:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.525+0000 7f5329bce640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5324103b60 0x7f532419f750 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f531c02c700 tx=0x7f531c034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.525+0000 7f531a7fc640 1 -- 172.21.15.26:0/1038552722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c039040 con 0x7f5324103b60 2024-04-21T10:30:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.525+0000 7f532b631640 1 -- 172.21.15.26:0/1038552722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53241a0050 con 0x7f5324103b60 2024-04-21T10:30:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.525+0000 7f531a7fc640 1 -- 172.21.15.26:0/1038552722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f531c044070 con 0x7f5324103b60 2024-04-21T10:30:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.525+0000 7f531a7fc640 1 -- 172.21.15.26:0/1038552722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c0042a0 con 0x7f5324103b60 2024-04-21T10:30:48.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.525+0000 7f532b631640 1 -- 172.21.15.26:0/1038552722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53241a0540 con 0x7f5324103b60 2024-04-21T10:30:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.526+0000 7f532b631640 1 -- 172.21.15.26:0/1038552722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53241092f0 con 0x7f5324103b60 2024-04-21T10:30:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.527+0000 7f531a7fc640 1 -- 172.21.15.26:0/1038552722 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f531c003970 con 0x7f5324103b60 2024-04-21T10:30:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.528+0000 7f531a7fc640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f53100a3bb0 0x7f53100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.528+0000 7f53293cd640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f53100a3bb0 0x7f53100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.528+0000 7f531a7fc640 1 -- 172.21.15.26:0/1038552722 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f531c0f3320 con 0x7f5324103b60 2024-04-21T10:30:48.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.528+0000 7f53293cd640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f53100a3bb0 0x7f53100a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5320005fd0 tx=0x7f5320005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:48.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.531+0000 7f531a7fc640 1 -- 172.21.15.26:0/1038552722 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f531c0bfcf0 con 0x7f5324103b60 2024-04-21T10:30:48.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:48 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3343812054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:48.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:48 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3281788869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:30:48.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:48 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/4111693026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:30:48.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:48 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3343812054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:48.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:48 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3281788869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:30:48.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:48 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/4111693026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:30:48.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.718+0000 7f532b631640 1 -- 172.21.15.26:0/1038552722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f532402d5c0 con 0x7f5324103b60 2024-04-21T10:30:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.718+0000 7f531a7fc640 1 -- 172.21.15.26:0/1038552722 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f531c0bf5f0 con 0x7f5324103b60 2024-04-21T10:30:48.718 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-21T10:30:48.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.720+0000 7f532b631640 1 -- 172.21.15.26:0/1038552722 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f53100a3bb0 msgr2=0x7f53100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.720+0000 7f532b631640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f53100a3bb0 0x7f53100a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5320005fd0 tx=0x7f5320005850 comp rx=0 tx=0).stop 2024-04-21T10:30:48.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.720+0000 7f532b631640 1 -- 172.21.15.26:0/1038552722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5324103b60 msgr2=0x7f532419f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.720+0000 7f532b631640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5324103b60 0x7f532419f750 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f531c02c700 tx=0x7f531c034760 comp rx=0 tx=0).stop 2024-04-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.720+0000 7f532b631640 1 -- 172.21.15.26:0/1038552722 shutdown_connections 2024-04-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.720+0000 7f532b631640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f53100a3bb0 0x7f53100a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.720+0000 7f532b631640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5324103b60 0x7f532419f750 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.720+0000 7f532b631640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53241031a0 0x7f53241986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.720+0000 7f532b631640 1 --2- 172.21.15.26:0/1038552722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5324101fb0 0x7f53241981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.720+0000 7f532b631640 1 -- 172.21.15.26:0/1038552722 >> 172.21.15.26:0/1038552722 conn(0x7f53240fd520 msgr2=0x7f53241064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.720+0000 7f532b631640 1 -- 172.21.15.26:0/1038552722 shutdown_connections 2024-04-21T10:30:48.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.721+0000 7f532b631640 1 -- 172.21.15.26:0/1038552722 wait complete. 2024-04-21T10:30:48.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:30:48.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:48 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3343812054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:48.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:48 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3281788869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:30:48.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:48 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/4111693026' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:30:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.833+0000 7f8af9b0e640 1 -- 172.21.15.26:0/1417893943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4101fc0 msgr2=0x7f8af4102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.833+0000 7f8af9b0e640 1 --2- 172.21.15.26:0/1417893943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4101fc0 0x7f8af4102440 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009900 tx=0x7f8ae802bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.833+0000 7f8af9b0e640 1 -- 172.21.15.26:0/1417893943 shutdown_connections 2024-04-21T10:30:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.833+0000 7f8af9b0e640 1 --2- 172.21.15.26:0/1417893943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af4102980 0x7f8af41092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.833+0000 7f8af9b0e640 1 --2- 172.21.15.26:0/1417893943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4101fc0 0x7f8af4102440 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.833+0000 7f8af9b0e640 1 --2- 172.21.15.26:0/1417893943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af4100e70 0x7f8af4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.833+0000 7f8af9b0e640 1 -- 172.21.15.26:0/1417893943 >> 172.21.15.26:0/1417893943 conn(0x7f8af40fc3e0 msgr2=0x7f8af40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.833+0000 7f8af9b0e640 1 -- 172.21.15.26:0/1417893943 shutdown_connections 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.833+0000 7f8af9b0e640 1 -- 172.21.15.26:0/1417893943 wait complete. 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af9b0e640 1 Processor -- start 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af9b0e640 1 -- start start 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af9b0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100e70 0x7f8af4069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af9b0e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af4101fc0 0x7f8af419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af9b0e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af4102980 0x7f8af419f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af9b0e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8af410c9a0 con 0x7f8af4100e70 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af9b0e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8af410c820 con 0x7f8af4101fc0 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af9b0e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8af410cb20 con 0x7f8af4102980 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100e70 0x7f8af4069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100e70 0x7f8af4069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58266/0 (socket says 172.21.15.26:58266) 2024-04-21T10:30:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af37fe640 1 -- 172.21.15.26:0/3948048150 learned_addr learned my addr 172.21.15.26:0/3948048150 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af2ffd640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af4101fc0 0x7f8af419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af3fff640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af4102980 0x7f8af419f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af37fe640 1 -- 172.21.15.26:0/3948048150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af4102980 msgr2=0x7f8af419f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.834+0000 7f8af37fe640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af4102980 0x7f8af419f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.835+0000 7f8af37fe640 1 -- 172.21.15.26:0/3948048150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af4101fc0 msgr2=0x7f8af419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.835+0000 7f8af37fe640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af4101fc0 0x7f8af419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.835+0000 7f8af37fe640 1 -- 172.21.15.26:0/3948048150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af419fe20 con 0x7f8af4100e70 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.835+0000 7f8af3fff640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af4102980 0x7f8af419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.835+0000 7f8af37fe640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100e70 0x7f8af4069c50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c890 tx=0x7f8ae400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.835+0000 7f8af0ff9640 1 -- 172.21.15.26:0/3948048150 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4004220 con 0x7f8af4100e70 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.835+0000 7f8af9b0e640 1 -- 172.21.15.26:0/3948048150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af41a00b0 con 0x7f8af4100e70 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.835+0000 7f8af0ff9640 1 -- 172.21.15.26:0/3948048150 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ae40043c0 con 0x7f8af4100e70 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.835+0000 7f8af0ff9640 1 -- 172.21.15.26:0/3948048150 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4004a00 con 0x7f8af4100e70 2024-04-21T10:30:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.835+0000 7f8af9b0e640 1 -- 172.21.15.26:0/3948048150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af41a05d0 con 0x7f8af4100e70 2024-04-21T10:30:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.836+0000 7f8af9b0e640 1 -- 172.21.15.26:0/3948048150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af4108250 con 0x7f8af4100e70 2024-04-21T10:30:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.837+0000 7f8af0ff9640 1 -- 172.21.15.26:0/3948048150 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8ae40079b0 con 0x7f8af4100e70 2024-04-21T10:30:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.838+0000 7f8af0ff9640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8adc0a3bb0 0x7f8adc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.838+0000 7f8af2ffd640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8adc0a3bb0 0x7f8adc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.838+0000 7f8af0ff9640 1 -- 172.21.15.26:0/3948048150 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f8ae40d05c0 con 0x7f8af4100e70 2024-04-21T10:30:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.839+0000 7f8af2ffd640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8adc0a3bb0 0x7f8adc0a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8005ab0 tx=0x7f8ae8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:48.842+0000 7f8af0ff9640 1 -- 172.21.15.26:0/3948048150 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ae4014030 con 0x7f8af4100e70 2024-04-21T10:30:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.071+0000 7f8af9b0e640 1 -- 172.21.15.26:0/3948048150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8af402d5c0 con 0x7f8af4100e70 2024-04-21T10:30:49.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.072+0000 7f8af0ff9640 1 -- 172.21.15.26:0/3948048150 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11884 (secure 0 0 0) 0x7f8ae409cf30 con 0x7f8af4100e70 2024-04-21T10:30:49.072 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:49.072 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:29:05.806617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24478,24488,24496]},"id":1}]} 2024-04-21T10:30:49.072 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 20 2024-04-21T10:30:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.074+0000 7f8af9b0e640 1 -- 172.21.15.26:0/3948048150 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8adc0a3bb0 msgr2=0x7f8adc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.074+0000 7f8af9b0e640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8adc0a3bb0 0x7f8adc0a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8005ab0 tx=0x7f8ae8005fd0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.074+0000 7f8af9b0e640 1 -- 172.21.15.26:0/3948048150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100e70 msgr2=0x7f8af4069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.074+0000 7f8af9b0e640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100e70 0x7f8af4069c50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c890 tx=0x7f8ae400cd50 comp rx=0 tx=0).stop 2024-04-21T10:30:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.074+0000 7f8af9b0e640 1 -- 172.21.15.26:0/3948048150 shutdown_connections 2024-04-21T10:30:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.074+0000 7f8af9b0e640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8adc0a3bb0 0x7f8adc0a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.074+0000 7f8af9b0e640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8af4102980 0x7f8af419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.074+0000 7f8af9b0e640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af4101fc0 0x7f8af419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.074+0000 7f8af9b0e640 1 --2- 172.21.15.26:0/3948048150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8af4100e70 0x7f8af4069c50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.074+0000 7f8af9b0e640 1 -- 172.21.15.26:0/3948048150 >> 172.21.15.26:0/3948048150 conn(0x7f8af40fc3e0 msgr2=0x7f8af4105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.074+0000 7f8af9b0e640 1 -- 172.21.15.26:0/3948048150 shutdown_connections 2024-04-21T10:30:49.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.074+0000 7f8af9b0e640 1 -- 172.21.15.26:0/3948048150 wait complete. 2024-04-21T10:30:49.085 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:30:49.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.187+0000 7fec0532c640 1 -- 172.21.15.26:0/3059430387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec00102ae0 msgr2=0x7fec00104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.187+0000 7fec0532c640 1 --2- 172.21.15.26:0/3059430387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec00102ae0 0x7fec00104ed0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7febf0009900 tx=0x7febf002bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:49.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.187+0000 7fec0532c640 1 -- 172.21.15.26:0/3059430387 shutdown_connections 2024-04-21T10:30:49.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.187+0000 7fec0532c640 1 --2- 172.21.15.26:0/3059430387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec00107d40 0x7fec0010a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.187+0000 7fec0532c640 1 --2- 172.21.15.26:0/3059430387 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec00105410 0x7fec00107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.187+0000 7fec0532c640 1 --2- 172.21.15.26:0/3059430387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec00102ae0 0x7fec00104ed0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.187+0000 7fec0532c640 1 -- 172.21.15.26:0/3059430387 >> 172.21.15.26:0/3059430387 conn(0x7fec000fc470 msgr2=0x7fec000fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.187+0000 7fec0532c640 1 -- 172.21.15.26:0/3059430387 shutdown_connections 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.187+0000 7fec0532c640 1 -- 172.21.15.26:0/3059430387 wait complete. 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7fec0532c640 1 Processor -- start 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7fec0532c640 1 -- start start 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7fec0532c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec00102ae0 0x7fec00198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7fec0532c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec00105410 0x7fec00198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7fec0532c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec00107d40 0x7fec0019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7fec0532c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fec0010db20 con 0x7fec00102ae0 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7fec0532c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec0010d9a0 con 0x7fec00105410 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7fec0532c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fec0010dca0 con 0x7fec00107d40 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7febfeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec00102ae0 0x7fec00198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7febfeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec00102ae0 0x7fec00198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58284/0 (socket says 172.21.15.26:58284) 2024-04-21T10:30:49.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7febfeffd640 1 -- 172.21.15.26:0/456823547 learned_addr learned my addr 172.21.15.26:0/456823547 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7febff7fe640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec00107d40 0x7fec0019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7febfe7fc640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec00105410 0x7fec00198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7febfeffd640 1 -- 172.21.15.26:0/456823547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec00107d40 msgr2=0x7fec0019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7febfeffd640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec00107d40 0x7fec0019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7febfeffd640 1 -- 172.21.15.26:0/456823547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec00105410 msgr2=0x7fec00198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7febfeffd640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec00105410 0x7fec00198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.188+0000 7febfeffd640 1 -- 172.21.15.26:0/456823547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0019fd00 con 0x7fec00102ae0 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.189+0000 7febfeffd640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec00102ae0 0x7fec00198130 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7febf0036040 tx=0x7febf00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.189+0000 7febe7fff640 1 -- 172.21.15.26:0/456823547 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0043020 con 0x7fec00102ae0 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.189+0000 7febe7fff640 1 -- 172.21.15.26:0/456823547 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf00054c0 con 0x7fec00102ae0 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.189+0000 7fec0532c640 1 -- 172.21.15.26:0/456823547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0019ff90 con 0x7fec00102ae0 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.189+0000 7febe7fff640 1 -- 172.21.15.26:0/456823547 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0003db0 con 0x7fec00102ae0 2024-04-21T10:30:49.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.189+0000 7fec0532c640 1 -- 172.21.15.26:0/456823547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec001a03f0 con 0x7fec00102ae0 2024-04-21T10:30:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.190+0000 7fec0532c640 1 -- 172.21.15.26:0/456823547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec000fdf60 con 0x7fec00102ae0 2024-04-21T10:30:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.191+0000 7febe7fff640 1 -- 172.21.15.26:0/456823547 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7febf0046020 con 0x7fec00102ae0 2024-04-21T10:30:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.191+0000 7febe7fff640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7febe80a3bb0 0x7febe80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.191+0000 7febfe7fc640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7febe80a3bb0 0x7febe80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.191+0000 7febe7fff640 1 -- 172.21.15.26:0/456823547 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7febf00f29b0 con 0x7fec00102ae0 2024-04-21T10:30:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.192+0000 7febfe7fc640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7febe80a3bb0 0x7febe80a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7febf4005e10 tx=0x7febf4005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.196+0000 7febe7fff640 1 -- 172.21.15.26:0/456823547 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7febf004d020 con 0x7fec00102ae0 2024-04-21T10:30:49.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.387+0000 7fec0532c640 1 -- 172.21.15.26:0/456823547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec00062b80 con 0x7fec00102ae0 2024-04-21T10:30:49.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.389+0000 7febe7fff640 1 -- 172.21.15.26:0/456823547 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22705 (secure 0 0 0) 0x7febf00bf380 con 0x7fec00102ae0 2024-04-21T10:30:49.389 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:30:49.389 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:30:03.297428+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:30:49.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.391+0000 7fec0532c640 1 -- 172.21.15.26:0/456823547 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7febe80a3bb0 msgr2=0x7febe80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.391+0000 7fec0532c640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7febe80a3bb0 0x7febe80a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7febf4005e10 tx=0x7febf4005da0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.391+0000 7fec0532c640 1 -- 172.21.15.26:0/456823547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec00102ae0 msgr2=0x7fec00198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.391+0000 7fec0532c640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec00102ae0 0x7fec00198130 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7febf0036040 tx=0x7febf00042f0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.391+0000 7fec0532c640 1 -- 172.21.15.26:0/456823547 shutdown_connections 2024-04-21T10:30:49.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.391+0000 7fec0532c640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7febe80a3bb0 0x7febe80a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.391+0000 7fec0532c640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec00107d40 0x7fec0019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.391+0000 7fec0532c640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec00105410 0x7fec00198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.391+0000 7fec0532c640 1 --2- 172.21.15.26:0/456823547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec00102ae0 0x7fec00198130 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.391+0000 7fec0532c640 1 -- 172.21.15.26:0/456823547 >> 172.21.15.26:0/456823547 conn(0x7fec000fc470 msgr2=0x7fec00103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.391+0000 7fec0532c640 1 -- 172.21.15.26:0/456823547 shutdown_connections 2024-04-21T10:30:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.391+0000 7fec0532c640 1 -- 172.21.15.26:0/456823547 wait complete. 2024-04-21T10:30:49.403 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T10:29:01.278914+0000', 'modified': '2024-04-21T10:29:05.806617+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24496, 'mds_1': 24488, 'mds_2': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6835/1594119566', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 1594119566}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 1594119566}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6837/2325983283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 2325983283}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 2325983283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6835/1573483809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 1573483809}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 1573483809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.190:6837/2091448511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 2091448511}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 2091448511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 1, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6839/2428626465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 2428626465}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 2428626465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.190:6839/1574521545', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 1574521545}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 1574521545}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24496, 'qdb_cluster': [24478, 24488, 24496]} 2024-04-21T10:30:49.403 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-21T10:30:49.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-21T10:30:49.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.506+0000 7f21eb5a4640 1 -- 172.21.15.26:0/3859715775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e4102a30 msgr2=0x7f21e4104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.506+0000 7f21eb5a4640 1 --2- 172.21.15.26:0/3859715775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e4102a30 0x7f21e4104e20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009900 tx=0x7f21e002bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:49.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.506+0000 7f21eb5a4640 1 -- 172.21.15.26:0/3859715775 shutdown_connections 2024-04-21T10:30:49.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.506+0000 7f21eb5a4640 1 --2- 172.21.15.26:0/3859715775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21e4107c90 0x7f21e410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.506+0000 7f21eb5a4640 1 --2- 172.21.15.26:0/3859715775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e4105360 0x7f21e4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.506+0000 7f21eb5a4640 1 --2- 172.21.15.26:0/3859715775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e4102a30 0x7f21e4104e20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.506+0000 7f21eb5a4640 1 -- 172.21.15.26:0/3859715775 >> 172.21.15.26:0/3859715775 conn(0x7f21e40fc440 msgr2=0x7f21e40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.506+0000 7f21eb5a4640 1 -- 172.21.15.26:0/3859715775 shutdown_connections 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.506+0000 7f21eb5a4640 1 -- 172.21.15.26:0/3859715775 wait complete. 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.507+0000 7f21eb5a4640 1 Processor -- start 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.507+0000 7f21eb5a4640 1 -- start start 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.507+0000 7f21eb5a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e4102a30 0x7f21e41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.507+0000 7f21eb5a4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21e4105360 0x7f21e4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.507+0000 7f21eb5a4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e4107c90 0x7f21e419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.507+0000 7f21eb5a4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21e410db40 con 0x7f21e4102a30 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.507+0000 7f21eb5a4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21e410d9c0 con 0x7f21e4107c90 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.507+0000 7f21eb5a4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f21e410dcc0 con 0x7f21e4105360 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.507+0000 7f21e9340640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e4102a30 0x7f21e41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.507+0000 7f21e9340640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e4102a30 0x7f21e41981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58300/0 (socket says 172.21.15.26:58300) 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.507+0000 7f21e9340640 1 -- 172.21.15.26:0/1647844721 learned_addr learned my addr 172.21.15.26:0/1647844721 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.507+0000 7f21e9b41640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e4107c90 0x7f21e419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21e8b3f640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21e4105360 0x7f21e4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21e9340640 1 -- 172.21.15.26:0/1647844721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21e4105360 msgr2=0x7f21e4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21e9340640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21e4105360 0x7f21e4198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21e9340640 1 -- 172.21.15.26:0/1647844721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e4107c90 msgr2=0x7f21e419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21e9340640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e4107c90 0x7f21e419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21e9340640 1 -- 172.21.15.26:0/1647844721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e419fde0 con 0x7f21e4102a30 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21e8b3f640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21e4105360 0x7f21e4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21e9340640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e4102a30 0x7f21e41981c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f21e0000c00 tx=0x7f21e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21da7fc640 1 -- 172.21.15.26:0/1647844721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0043020 con 0x7f21e4102a30 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21da7fc640 1 -- 172.21.15.26:0/1647844721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e0004aa0 con 0x7f21e4102a30 2024-04-21T10:30:49.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21eb5a4640 1 -- 172.21.15.26:0/1647844721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e41a0070 con 0x7f21e4102a30 2024-04-21T10:30:49.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21da7fc640 1 -- 172.21.15.26:0/1647844721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0003d70 con 0x7f21e4102a30 2024-04-21T10:30:49.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.508+0000 7f21eb5a4640 1 -- 172.21.15.26:0/1647844721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e41a05c0 con 0x7f21e4102a30 2024-04-21T10:30:49.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.510+0000 7f21da7fc640 1 -- 172.21.15.26:0/1647844721 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f21e0046020 con 0x7f21e4102a30 2024-04-21T10:30:49.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21ee198740 1 Processor -- start 2024-04-21T10:30:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21ee198740 1 -- start start 2024-04-21T10:30:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21ee198740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f07529d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21ee198740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529dc40 0x55f07529e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21ee198740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21ee198740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f0751fdb10 con 0x55f07529e6e0 2024-04-21T10:30:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21ee198740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f07519b240 con 0x55f07529d250 2024-04-21T10:30:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21ee198740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f0751f9ad0 con 0x55f07529dc40 2024-04-21T10:30:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21e8b3f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21e8b3f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58314/0 (socket says 172.21.15.26:58314) 2024-04-21T10:30:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21e8b3f640 1 -- 172.21.15.26:0/2364706474 learned_addr learned my addr 172.21.15.26:0/2364706474 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21da7fc640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f21d00a3bb0 0x7f21d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21e9b41640 1 --2- 172.21.15.26:0/2364706474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f07529d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21e9340640 1 --2- 172.21.15.26:0/2364706474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529dc40 0x55f07529e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21e9b41640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f21d00a3bb0 0x7f21d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21e8b3f640 1 -- 172.21.15.26:0/2364706474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529dc40 msgr2=0x55f07529e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21e8b3f640 1 --2- 172.21.15.26:0/2364706474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529dc40 0x55f07529e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21e8b3f640 1 -- 172.21.15.26:0/2364706474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 msgr2=0x55f07529d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21e8b3f640 1 --2- 172.21.15.26:0/2364706474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f07529d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21e8b3f640 1 -- 172.21.15.26:0/2364706474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f074f102d0 con 0x55f07529e6e0 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.511+0000 7f21da7fc640 1 -- 172.21.15.26:0/1647844721 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f21e00f2760 con 0x7f21e4102a30 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.512+0000 7f21e8b3f640 1 --2- 172.21.15.26:0/2364706474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752a52c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009cb0 tx=0x7f21d40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5cbb239a9299bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.512+0000 7f21e9b41640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f21d00a3bb0 0x7f21d00a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005c40 tx=0x7f21dc007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.512+0000 7f21cf7fe640 1 -- 172.21.15.26:0/2364706474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4004020 con 0x55f07529e6e0 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.512+0000 7f21cf7fe640 1 -- 172.21.15.26:0/2364706474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d40041c0 con 0x55f07529e6e0 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.512+0000 7f21cf7fe640 1 -- 172.21.15.26:0/2364706474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4012780 con 0x55f07529e6e0 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.512+0000 7f21ee198740 1 -- 172.21.15.26:0/2364706474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 msgr2=0x55f0752a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.512+0000 7f21ee198740 1 --2- 172.21.15.26:0/2364706474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752a52c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009cb0 tx=0x7f21d40117b0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 -- 172.21.15.26:0/2364706474 shutdown_connections 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 --2- 172.21.15.26:0/2364706474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752a52c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 --2- 172.21.15.26:0/2364706474 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529dc40 0x55f07529e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 --2- 172.21.15.26:0/2364706474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f07529d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 -- 172.21.15.26:0/2364706474 >> 172.21.15.26:0/2364706474 conn(0x55f075294f40 msgr2=0x55f075293490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 -- 172.21.15.26:0/2364706474 shutdown_connections 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 -- 172.21.15.26:0/2364706474 wait complete. 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 -- start start 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 Processor -- start 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529d250 0x55f0752a7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529dc40 0x55f0752a77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f0752a58c0 con 0x55f07529e6e0 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f075186190 con 0x55f07529dc40 2024-04-21T10:30:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f0752a5a40 con 0x55f07529d250 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21ee198740 10 client.0 _ll_register_callbacks cb 0x55f0751818b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.513+0000 7f21e9b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21e9b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58318/0 (socket says 172.21.15.26:58318) 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21e9b41640 1 -- 172.21.15.26:0/3494810048 learned_addr learned my addr 172.21.15.26:0/3494810048 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21e9340640 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529d250 0x55f0752a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21e8b3f640 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529dc40 0x55f0752a77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21e9b41640 1 -- 172.21.15.26:0/3494810048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529d250 msgr2=0x55f0752a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21e9b41640 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529d250 0x55f0752a7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21e9b41640 1 -- 172.21.15.26:0/3494810048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529dc40 msgr2=0x55f0752a77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21e9b41640 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529dc40 0x55f0752a77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21e9b41640 1 -- 172.21.15.26:0/3494810048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f074fda6b0 con 0x55f07529e6e0 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21e8b3f640 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529dc40 0x55f0752a77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21e9b41640 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752ae7e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0045f0 tx=0x7f21dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f05b64624252d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21ccff9640 1 -- 172.21.15.26:0/3494810048 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc024020 con 0x55f07529e6e0 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21ccff9640 1 -- 172.21.15.26:0/3494810048 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f21dc00f450 con 0x55f07529e6e0 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21ee198740 1 -- 172.21.15.26:0/3494810048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d40041c0 con 0x55f07529e6e0 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21ccff9640 1 -- 172.21.15.26:0/3494810048 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f21dc02f050 con 0x55f07529e6e0 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21ee198740 10 client.15222 fetch_fsmap learned FSMap version 20 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21ee198740 1 -- 172.21.15.26:0/3494810048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f0752b1bc0 con 0x55f07529e6e0 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.514+0000 7f21ccff9640 1 -- 172.21.15.26:0/3494810048 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d40041c0 con 0x55f07529e6e0 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.515+0000 7f21ccff9640 1 -- 172.21.15.26:0/3494810048 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f21dc020780 con 0x55f07529e6e0 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.515+0000 7f21ee198740 10 client.15222 fetch_fsmap finished waiting for FSMap version 20 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.515+0000 7f21ee198740 10 client.15222 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.515+0000 7f21ee198740 20 client.15222 populate_metadata read hostname 'smithi026' 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.515+0000 7f21ee198740 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x55f0752b4310 0x55f0752b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.515+0000 7f21ee198740 4 client.15222 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.515+0000 7f21ee198740 1 -- 172.21.15.26:0/3494810048 --> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f07520b530 con 0x55f0752b4310 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.515+0000 7f21e9340640 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x55f0752b4310 0x55f0752b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.515+0000 7f21e9340640 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x55f0752b4310 0x55f0752b5780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b33c1c859220833 server_cookie=4909351a52d6f907 in_seq=0 out_seq=0 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.515+0000 7f21ccff9640 10 client.15222 ms_handle_connect on v2:172.21.15.190:6836/2091448511 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.516+0000 7f21ccff9640 1 -- 172.21.15.26:0/3494810048 <== mds.2 v2:172.21.15.190:6836/2091448511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f07520b530 con 0x55f0752b4310 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.516+0000 7f21ccff9640 10 client.15222 handle_command_reply: tid=0 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.516+0000 7f21ee198740 1 client.15222 shutdown 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.517+0000 7f21ee198740 1 -- 172.21.15.26:0/3494810048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 msgr2=0x55f0752ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.517+0000 7f21ee198740 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752ae7e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0045f0 tx=0x7f21dc004a40 comp rx=0 tx=0).stop 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.517+0000 7f21ee198740 1 -- 172.21.15.26:0/3494810048 shutdown_connections 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.517+0000 7f21ee198740 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x55f0752b4310 0x55f0752b5780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.517+0000 7f21ee198740 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752ae7e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.517+0000 7f21ee198740 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529dc40 0x55f0752a77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.517+0000 7f21ccff9640 0 client.15222 ms_handle_reset on v2:172.21.15.190:6836/2091448511 2024-04-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.517+0000 7f21ee198740 1 --2- 172.21.15.26:0/3494810048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529d250 0x55f0752a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.517+0000 7f21ee198740 1 -- 172.21.15.26:0/3494810048 >> 172.21.15.26:0/3494810048 conn(0x55f075294f40 msgr2=0x55f075292a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.517+0000 7f21ee198740 1 -- 172.21.15.26:0/3494810048 shutdown_connections 2024-04-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.517+0000 7f21ee198740 1 -- 172.21.15.26:0/3494810048 wait complete. 2024-04-21T10:30:49.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.517+0000 7f21ee198740 20 client.15222 trim_cache size 0 max 16384 2024-04-21T10:30:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.535+0000 7f21ee198740 1 Processor -- start 2024-04-21T10:30:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.535+0000 7f21ee198740 1 -- start start 2024-04-21T10:30:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.535+0000 7f21ee198740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f075293460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.535+0000 7f21ee198740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529dc40 0x55f0752939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.535+0000 7f21ee198740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529e6e0 0x55f0752ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.535+0000 7f21ee198740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f0752b3ac0 con 0x55f07529dc40 2024-04-21T10:30:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.535+0000 7f21ee198740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f0752b4190 con 0x55f07529d250 2024-04-21T10:30:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.535+0000 7f21ee198740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f0752b4860 con 0x55f07529e6e0 2024-04-21T10:30:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.535+0000 7f21e8b3f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529dc40 0x55f0752939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.535+0000 7f21e8b3f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529dc40 0x55f0752939a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58322/0 (socket says 172.21.15.26:58322) 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.535+0000 7f21e8b3f640 1 -- 172.21.15.26:0/2710587526 learned_addr learned my addr 172.21.15.26:0/2710587526 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.536+0000 7f21e9340640 1 --2- 172.21.15.26:0/2710587526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f075293460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.536+0000 7f21e8b3f640 1 -- 172.21.15.26:0/2710587526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529e6e0 msgr2=0x55f0752ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.536+0000 7f21e8b3f640 1 --2- 172.21.15.26:0/2710587526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529e6e0 0x55f0752ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.536+0000 7f21e8b3f640 1 -- 172.21.15.26:0/2710587526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 msgr2=0x55f075293460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.536+0000 7f21e8b3f640 1 --2- 172.21.15.26:0/2710587526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f075293460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.536+0000 7f21e8b3f640 1 -- 172.21.15.26:0/2710587526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0752a83e0 con 0x55f07529dc40 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.536+0000 7f21e9340640 1 --2- 172.21.15.26:0/2710587526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f075293460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.536+0000 7f21e8b3f640 1 --2- 172.21.15.26:0/2710587526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529dc40 0x55f0752939a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009cb0 tx=0x7f21d4009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd99f83bd896ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.536+0000 7f21ceffd640 1 -- 172.21.15.26:0/2710587526 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d40097a0 con 0x55f07529dc40 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.536+0000 7f21ceffd640 1 -- 172.21.15.26:0/2710587526 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d40054e0 con 0x55f07529dc40 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.536+0000 7f21ceffd640 1 -- 172.21.15.26:0/2710587526 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4012450 con 0x55f07529dc40 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 -- 172.21.15.26:0/2710587526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529dc40 msgr2=0x55f0752939a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 --2- 172.21.15.26:0/2710587526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529dc40 0x55f0752939a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009cb0 tx=0x7f21d4009e50 comp rx=0 tx=0).stop 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 -- 172.21.15.26:0/2710587526 shutdown_connections 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 --2- 172.21.15.26:0/2710587526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529e6e0 0x55f0752ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 --2- 172.21.15.26:0/2710587526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529dc40 0x55f0752939a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 --2- 172.21.15.26:0/2710587526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f075293460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 -- 172.21.15.26:0/2710587526 >> 172.21.15.26:0/2710587526 conn(0x55f075294f40 msgr2=0x55f0752b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 -- 172.21.15.26:0/2710587526 shutdown_connections 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 -- 172.21.15.26:0/2710587526 wait complete. 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 -- start start 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 Processor -- start 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f0752a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529dc40 0x55f0752aa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f0752b5600 con 0x55f07529e6e0 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f0752b4f30 con 0x55f07529d250 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f0752b5cd0 con 0x55f07529dc40 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21ee198740 10 client.0 _ll_register_callbacks cb 0x7f21dc007b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.537+0000 7f21e9b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21e9b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752ad410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58332/0 (socket says 172.21.15.26:58332) 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21e9b41640 1 -- 172.21.15.26:0/3981014083 learned_addr learned my addr 172.21.15.26:0/3981014083 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21e8b3f640 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529dc40 0x55f0752aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21e9340640 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f0752a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21e9b41640 1 -- 172.21.15.26:0/3981014083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529dc40 msgr2=0x55f0752aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21e9b41640 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529dc40 0x55f0752aa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21e9b41640 1 -- 172.21.15.26:0/3981014083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 msgr2=0x55f0752a9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21e9b41640 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f0752a9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21e9b41640 1 -- 172.21.15.26:0/3981014083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0752ada00 con 0x55f07529e6e0 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21e9b41640 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752ad410 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f21dc020de0 tx=0x7f21dc005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62010a465c907701 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21ccff9640 1 -- 172.21.15.26:0/3981014083 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc020e60 con 0x55f07529e6e0 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21ccff9640 1 -- 172.21.15.26:0/3981014083 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f21dc020420 con 0x55f07529e6e0 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21ee198740 1 -- 172.21.15.26:0/3981014083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d40054e0 con 0x55f07529e6e0 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21ccff9640 1 -- 172.21.15.26:0/3981014083 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f21dc024040 con 0x55f07529e6e0 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21ee198740 10 client.15228 fetch_fsmap learned FSMap version 20 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.538+0000 7f21ee198740 1 -- 172.21.15.26:0/3981014083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f07531a2c0 con 0x55f07529e6e0 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21ccff9640 1 -- 172.21.15.26:0/3981014083 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d40054e0 con 0x55f07529e6e0 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21ccff9640 1 -- 172.21.15.26:0/3981014083 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f21dc013ac0 con 0x55f07529e6e0 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21ee198740 10 client.15228 fetch_fsmap finished waiting for FSMap version 20 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21ee198740 10 client.15228 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21ee198740 20 client.15228 populate_metadata read hostname 'smithi026' 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21ee198740 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x55f07531ed60 0x55f07531da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21ee198740 4 client.15228 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21ee198740 1 -- 172.21.15.26:0/3981014083 --> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] -- command(tid 0: {"prefix": "status"}) -- 0x55f0752053a0 con 0x55f07531ed60 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21e9340640 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x55f07531ed60 0x55f07531da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21ccff9640 10 client.15228 ms_handle_connect on v2:172.21.15.190:6836/2091448511 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21e9340640 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x55f07531ed60 0x55f07531da70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d03ac171c1fca709 server_cookie=6b08a5b45d4d7d1 in_seq=0 out_seq=0 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21ccff9640 1 -- 172.21.15.26:0/3981014083 <== mds.2 v2:172.21.15.190:6836/2091448511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55f0752053a0 con 0x55f07531ed60 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21ccff9640 10 client.15228 handle_command_reply: tid=0 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.539+0000 7f21ee198740 1 client.15228 shutdown 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21ee198740 1 -- 172.21.15.26:0/3981014083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 msgr2=0x55f0752ad410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21ee198740 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752ad410 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f21dc020de0 tx=0x7f21dc005a50 comp rx=0 tx=0).stop 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21ee198740 1 -- 172.21.15.26:0/3981014083 shutdown_connections 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21ee198740 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x55f07531ed60 0x55f07531da70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21ee198740 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f07529e6e0 0x55f0752ad410 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21ee198740 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07529dc40 0x55f0752aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21ee198740 1 --2- 172.21.15.26:0/3981014083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f07529d250 0x55f0752a9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21ee198740 1 -- 172.21.15.26:0/3981014083 >> 172.21.15.26:0/3981014083 conn(0x55f075294f40 msgr2=0x55f0752a1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21ccff9640 0 client.15228 ms_handle_reset on v2:172.21.15.190:6836/2091448511 2024-04-21T10:30:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21ee198740 1 -- 172.21.15.26:0/3981014083 shutdown_connections 2024-04-21T10:30:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21ee198740 1 -- 172.21.15.26:0/3981014083 wait complete. 2024-04-21T10:30:49.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21ee198740 20 client.15228 trim_cache size 0 max 16384 2024-04-21T10:30:49.546 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:30:49.546 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:30:49.546 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-04-21T10:30:49.546 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24478, 2024-04-21T10:30:49.546 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-21T10:30:49.546 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-21T10:30:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-21T10:30:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 104.737326272, 2024-04-21T10:30:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 20, 2024-04-21T10:30:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 89, 2024-04-21T10:30:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 87, 2024-04-21T10:30:49.547 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 127.055000948 2024-04-21T10:30:49.547 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:30:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21eb5a4640 1 -- 172.21.15.26:0/1647844721 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f21d00a3bb0 msgr2=0x7f21d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21eb5a4640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f21d00a3bb0 0x7f21d00a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005c40 tx=0x7f21dc007520 comp rx=0 tx=0).stop 2024-04-21T10:30:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21eb5a4640 1 -- 172.21.15.26:0/1647844721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e4102a30 msgr2=0x7f21e41981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21eb5a4640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e4102a30 0x7f21e41981c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f21e0000c00 tx=0x7f21e00042f0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21eb5a4640 1 -- 172.21.15.26:0/1647844721 shutdown_connections 2024-04-21T10:30:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21eb5a4640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f21d00a3bb0 0x7f21d00a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21eb5a4640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21e4107c90 0x7f21e419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21eb5a4640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21e4105360 0x7f21e4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21eb5a4640 1 --2- 172.21.15.26:0/1647844721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e4102a30 0x7f21e41981c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21eb5a4640 1 -- 172.21.15.26:0/1647844721 >> 172.21.15.26:0/1647844721 conn(0x7f21e40fc440 msgr2=0x7f21e4103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21eb5a4640 1 -- 172.21.15.26:0/1647844721 shutdown_connections 2024-04-21T10:30:49.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.540+0000 7f21eb5a4640 1 -- 172.21.15.26:0/1647844721 wait complete. 2024-04-21T10:30:49.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T10:30:49.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.653+0000 7f14ce94a640 1 -- 172.21.15.26:0/3507196860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14c81009e0 msgr2=0x7f14c8100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.653+0000 7f14ce94a640 1 --2- 172.21.15.26:0/3507196860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14c81009e0 0x7f14c8100e90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009ca0 tx=0x7f14c002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.653+0000 7f14ce94a640 1 -- 172.21.15.26:0/3507196860 shutdown_connections 2024-04-21T10:30:49.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.653+0000 7f14ce94a640 1 --2- 172.21.15.26:0/3507196860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14c81009e0 0x7f14c8100e90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.653+0000 7f14ce94a640 1 --2- 172.21.15.26:0/3507196860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c8100020 0x7f14c81004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.653+0000 7f14ce94a640 1 --2- 172.21.15.26:0/3507196860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c8102530 0x7f14c8102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.653+0000 7f14ce94a640 1 -- 172.21.15.26:0/3507196860 >> 172.21.15.26:0/3507196860 conn(0x7f14c80fbb80 msgr2=0x7f14c80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.653+0000 7f14ce94a640 1 -- 172.21.15.26:0/3507196860 shutdown_connections 2024-04-21T10:30:49.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.653+0000 7f14ce94a640 1 -- 172.21.15.26:0/3507196860 wait complete. 2024-04-21T10:30:49.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.654+0000 7f14ce94a640 1 Processor -- start 2024-04-21T10:30:49.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.654+0000 7f14ce94a640 1 -- start start 2024-04-21T10:30:49.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.654+0000 7f14ce94a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c8100020 0x7f14c819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.654+0000 7f14ce94a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c81009e0 0x7f14c819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.654+0000 7f14ce94a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14c8102530 0x7f14c81a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.654+0000 7f14ce94a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14c810fcd0 con 0x7f14c8102530 2024-04-21T10:30:49.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.654+0000 7f14ce94a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14c810fb50 con 0x7f14c81009e0 2024-04-21T10:30:49.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.654+0000 7f14ce94a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f14c810fe50 con 0x7f14c8100020 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.654+0000 7f14ccee7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14c8102530 0x7f14c81a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.654+0000 7f14ccee7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14c8102530 0x7f14c81a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58360/0 (socket says 172.21.15.26:58360) 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.654+0000 7f14ccee7640 1 -- 172.21.15.26:0/766364482 learned_addr learned my addr 172.21.15.26:0/766364482 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.654+0000 7f14c77fe640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c81009e0 0x7f14c819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.655+0000 7f14c7fff640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c8100020 0x7f14c819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.655+0000 7f14ccee7640 1 -- 172.21.15.26:0/766364482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c8100020 msgr2=0x7f14c819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.655+0000 7f14ccee7640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c8100020 0x7f14c819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.655+0000 7f14ccee7640 1 -- 172.21.15.26:0/766364482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c81009e0 msgr2=0x7f14c819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.655+0000 7f14ccee7640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c81009e0 0x7f14c819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.655+0000 7f14ccee7640 1 -- 172.21.15.26:0/766364482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c81a1f00 con 0x7f14c8102530 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.655+0000 7f14ccee7640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14c8102530 0x7f14c81a1890 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f14c002c700 tx=0x7f14c0034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.655+0000 7f14c57fa640 1 -- 172.21.15.26:0/766364482 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0039040 con 0x7f14c8102530 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.655+0000 7f14c57fa640 1 -- 172.21.15.26:0/766364482 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c0044070 con 0x7f14c8102530 2024-04-21T10:30:49.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.655+0000 7f14ce94a640 1 -- 172.21.15.26:0/766364482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c81a2190 con 0x7f14c8102530 2024-04-21T10:30:49.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.655+0000 7f14c57fa640 1 -- 172.21.15.26:0/766364482 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00042a0 con 0x7f14c8102530 2024-04-21T10:30:49.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.655+0000 7f14ce94a640 1 -- 172.21.15.26:0/766364482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c81a2680 con 0x7f14c8102530 2024-04-21T10:30:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.657+0000 7f14c57fa640 1 -- 172.21.15.26:0/766364482 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f14c0003970 con 0x7f14c8102530 2024-04-21T10:30:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14d160e740 1 Processor -- start 2024-04-21T10:30:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14d160e740 1 -- start start 2024-04-21T10:30:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14d160e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e3f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14d160e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3f830 0x55d4f4e3fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14d160e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e402d0 0x55d4f4e46eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14d160e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4da3b10 con 0x55d4f4e3ee40 2024-04-21T10:30:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14d160e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4d41240 con 0x55d4f4e3f830 2024-04-21T10:30:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14d160e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4d9fad0 con 0x55d4f4e402d0 2024-04-21T10:30:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14c57fa640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f14b40a3bb0 0x7f14b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e3f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e3f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58372/0 (socket says 172.21.15.26:58372) 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14c77fe640 1 -- 172.21.15.26:0/1299992455 learned_addr learned my addr 172.21.15.26:0/1299992455 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14ccee7640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f14b40a3bb0 0x7f14b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14c7fff640 1 --2- 172.21.15.26:0/1299992455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3f830 0x55d4f4e3fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14c57fa640 1 -- 172.21.15.26:0/766364482 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f14c00f27f0 con 0x7f14c8102530 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14c77fe640 1 --2- 172.21.15.26:0/1299992455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e402d0 0x55d4f4e46eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14c77fe640 1 -- 172.21.15.26:0/1299992455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e402d0 msgr2=0x55d4f4e46eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14c77fe640 1 --2- 172.21.15.26:0/1299992455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e402d0 0x55d4f4e46eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14c77fe640 1 -- 172.21.15.26:0/1299992455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3f830 msgr2=0x55d4f4e3fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14c77fe640 1 --2- 172.21.15.26:0/1299992455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3f830 0x55d4f4e3fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.658+0000 7f14c77fe640 1 -- 172.21.15.26:0/1299992455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4f4ab62d0 con 0x55d4f4e3ee40 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.659+0000 7f14c77fe640 1 --2- 172.21.15.26:0/1299992455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e3f260 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f14b800bfd0 tx=0x7f14b800b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658858b21ece69d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.659+0000 7f14ccee7640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f14b40a3bb0 0x7f14b40a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f14c00bb490 tx=0x7f14c0089110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.659+0000 7f14be7fc640 1 -- 172.21.15.26:0/1299992455 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8004020 con 0x55d4f4e3ee40 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.659+0000 7f14be7fc640 1 -- 172.21.15.26:0/1299992455 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b80041c0 con 0x55d4f4e3ee40 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.659+0000 7f14be7fc640 1 -- 172.21.15.26:0/1299992455 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b800f6e0 con 0x55d4f4e3ee40 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.659+0000 7f14d160e740 1 -- 172.21.15.26:0/1299992455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3ee40 msgr2=0x55d4f4e3f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.659+0000 7f14d160e740 1 --2- 172.21.15.26:0/1299992455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e3f260 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f14b800bfd0 tx=0x7f14b800b820 comp rx=0 tx=0).stop 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 -- 172.21.15.26:0/1299992455 shutdown_connections 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 --2- 172.21.15.26:0/1299992455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e402d0 0x55d4f4e46eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 --2- 172.21.15.26:0/1299992455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3f830 0x55d4f4e3fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 --2- 172.21.15.26:0/1299992455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e3f260 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 -- 172.21.15.26:0/1299992455 >> 172.21.15.26:0/1299992455 conn(0x55d4f4e3af40 msgr2=0x55d4f4e39490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 -- 172.21.15.26:0/1299992455 shutdown_connections 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 -- 172.21.15.26:0/1299992455 wait complete. 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 -- start start 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 Processor -- start 2024-04-21T10:30:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e48ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3f830 0x55d4f4e49400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e402d0 0x55d4f4e50440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4e474b0 con 0x55d4f4e3f830 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4d2c190 con 0x55d4f4e402d0 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4e47630 con 0x55d4f4e3ee40 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14d160e740 10 client.0 _ll_register_callbacks cb 0x55d4f4d278b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14c7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3f830 0x55d4f4e49400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14c7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3f830 0x55d4f4e49400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58376/0 (socket says 172.21.15.26:58376) 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.660+0000 7f14c7fff640 1 -- 172.21.15.26:0/3673078133 learned_addr learned my addr 172.21.15.26:0/3673078133 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14c77fe640 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e48ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14c77fe640 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e402d0 0x55d4f4e50440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14c7fff640 1 -- 172.21.15.26:0/3673078133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3ee40 msgr2=0x55d4f4e48ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14c7fff640 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e48ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14c7fff640 1 -- 172.21.15.26:0/3673078133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e402d0 msgr2=0x55d4f4e50440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14c7fff640 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e402d0 0x55d4f4e50440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14c7fff640 1 -- 172.21.15.26:0/3673078133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4f4b806b0 con 0x55d4f4e3f830 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14c7fff640 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3f830 0x55d4f4e49400 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f14b000cec0 tx=0x7f14b00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ce8dbc4e3e29f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14a77fe640 1 -- 172.21.15.26:0/3673078133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b00119f0 con 0x55d4f4e3f830 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14a77fe640 1 -- 172.21.15.26:0/3673078133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f14b0011b90 con 0x55d4f4e3f830 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14d160e740 1 -- 172.21.15.26:0/3673078133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b80041c0 con 0x55d4f4e3f830 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14a77fe640 1 -- 172.21.15.26:0/3673078133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f14b0011d90 con 0x55d4f4e3f830 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14d160e740 10 client.15240 fetch_fsmap learned FSMap version 20 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14d160e740 1 -- 172.21.15.26:0/3673078133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d4f4e53860 con 0x55d4f4e3f830 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.661+0000 7f14a77fe640 1 -- 172.21.15.26:0/3673078133 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b80041c0 con 0x55d4f4e3f830 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.662+0000 7f14a77fe640 1 -- 172.21.15.26:0/3673078133 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f14b0023050 con 0x55d4f4e3f830 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.662+0000 7f14d160e740 10 client.15240 fetch_fsmap finished waiting for FSMap version 20 2024-04-21T10:30:49.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.662+0000 7f14d160e740 10 client.15240 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.662+0000 7f14d160e740 20 client.15240 populate_metadata read hostname 'smithi026' 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.662+0000 7f14d160e740 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x55d4f4e55ef0 0x55d4f4e57360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.662+0000 7f14d160e740 4 client.15240 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.662+0000 7f14d160e740 1 -- 172.21.15.26:0/3673078133 --> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4f4db1530 con 0x55d4f4e55ef0 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.662+0000 7f14c77fe640 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x55d4f4e55ef0 0x55d4f4e57360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.662+0000 7f14a77fe640 10 client.15240 ms_handle_connect on v2:172.21.15.169:6838/2428626465 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.662+0000 7f14c77fe640 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x55d4f4e55ef0 0x55d4f4e57360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1092f59a7e2960d0 server_cookie=84a96fd10d3664a in_seq=0 out_seq=0 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.663+0000 7f14a77fe640 1 -- 172.21.15.26:0/3673078133 <== mds.1 v2:172.21.15.169:6838/2428626465 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4f4db1530 con 0x55d4f4e55ef0 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.663+0000 7f14a77fe640 10 client.15240 handle_command_reply: tid=0 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.663+0000 7f14d160e740 1 client.15240 shutdown 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.663+0000 7f14d160e740 1 -- 172.21.15.26:0/3673078133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3f830 msgr2=0x55d4f4e49400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.663+0000 7f14d160e740 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3f830 0x55d4f4e49400 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f14b000cec0 tx=0x7f14b00039c0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.663+0000 7f14d160e740 1 -- 172.21.15.26:0/3673078133 shutdown_connections 2024-04-21T10:30:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.663+0000 7f14d160e740 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x55d4f4e55ef0 0x55d4f4e57360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.663+0000 7f14d160e740 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e402d0 0x55d4f4e50440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.663+0000 7f14a77fe640 0 client.15240 ms_handle_reset on v2:172.21.15.169:6838/2428626465 2024-04-21T10:30:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.663+0000 7f14d160e740 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e3f830 0x55d4f4e49400 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.663+0000 7f14d160e740 1 --2- 172.21.15.26:0/3673078133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e48ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.663+0000 7f14d160e740 1 -- 172.21.15.26:0/3673078133 >> 172.21.15.26:0/3673078133 conn(0x55d4f4e3af40 msgr2=0x55d4f4e38a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.664+0000 7f14d160e740 1 -- 172.21.15.26:0/3673078133 shutdown_connections 2024-04-21T10:30:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.664+0000 7f14d160e740 1 -- 172.21.15.26:0/3673078133 wait complete. 2024-04-21T10:30:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.664+0000 7f14d160e740 20 client.15240 trim_cache size 0 max 16384 2024-04-21T10:30:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14d160e740 1 Processor -- start 2024-04-21T10:30:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14d160e740 1 -- start start 2024-04-21T10:30:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14d160e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e43840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14d160e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3f830 0x55d4f4e43d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14d160e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 0x55d4f4e50400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14d160e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4e556a0 con 0x55d4f4e402d0 2024-04-21T10:30:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14d160e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4e55d70 con 0x55d4f4e3f830 2024-04-21T10:30:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14d160e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4e56440 con 0x55d4f4e3ee40 2024-04-21T10:30:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 0x55d4f4e50400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 0x55d4f4e50400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58378/0 (socket says 172.21.15.26:58378) 2024-04-21T10:30:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14c77fe640 1 -- 172.21.15.26:0/1763808993 learned_addr learned my addr 172.21.15.26:0/1763808993 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14c77fe640 1 -- 172.21.15.26:0/1763808993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3ee40 msgr2=0x55d4f4e43840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14c77fe640 1 --2- 172.21.15.26:0/1763808993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e43840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14c77fe640 1 -- 172.21.15.26:0/1763808993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3f830 msgr2=0x55d4f4e43d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14c77fe640 1 --2- 172.21.15.26:0/1763808993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3f830 0x55d4f4e43d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14c77fe640 1 -- 172.21.15.26:0/1763808993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4f4e34e70 con 0x55d4f4e402d0 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14c77fe640 1 --2- 172.21.15.26:0/1763808993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 0x55d4f4e50400 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f14b800bfd0 tx=0x7f14b800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d138e803193000f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14bdffb640 1 -- 172.21.15.26:0/1763808993 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8027040 con 0x55d4f4e402d0 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14bdffb640 1 -- 172.21.15.26:0/1763808993 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b8004450 con 0x55d4f4e402d0 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.682+0000 7f14bdffb640 1 -- 172.21.15.26:0/1763808993 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8003830 con 0x55d4f4e402d0 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.683+0000 7f14d160e740 1 -- 172.21.15.26:0/1763808993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 msgr2=0x55d4f4e50400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.683+0000 7f14d160e740 1 --2- 172.21.15.26:0/1763808993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 0x55d4f4e50400 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f14b800bfd0 tx=0x7f14b800b5b0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.683+0000 7f14d160e740 1 -- 172.21.15.26:0/1763808993 shutdown_connections 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.683+0000 7f14d160e740 1 --2- 172.21.15.26:0/1763808993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 0x55d4f4e50400 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.683+0000 7f14d160e740 1 --2- 172.21.15.26:0/1763808993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3f830 0x55d4f4e43d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.683+0000 7f14d160e740 1 --2- 172.21.15.26:0/1763808993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e43840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.683+0000 7f14d160e740 1 -- 172.21.15.26:0/1763808993 >> 172.21.15.26:0/1763808993 conn(0x55d4f4e3af40 msgr2=0x55d4f4e39400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.683+0000 7f14d160e740 1 -- 172.21.15.26:0/1763808993 shutdown_connections 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.683+0000 7f14d160e740 1 -- 172.21.15.26:0/1763808993 wait complete. 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.683+0000 7f14d160e740 1 -- start start 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14d160e740 1 Processor -- start 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14d160e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14d160e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3f830 0x55d4f4e42c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14d160e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 0x55d4f4e4eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14d160e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4e571e0 con 0x55d4f4e402d0 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14d160e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4e56b10 con 0x55d4f4e3ee40 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14d160e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4e578b0 con 0x55d4f4e3f830 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14d160e740 10 client.0 _ll_register_callbacks cb 0x7f14b0004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 0x55d4f4e4eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 0x55d4f4e4eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58380/0 (socket says 172.21.15.26:58380) 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14c77fe640 1 -- 172.21.15.26:0/1866327690 learned_addr learned my addr 172.21.15.26:0/1866327690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14c77fe640 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14c7fff640 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3f830 0x55d4f4e42c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14c77fe640 1 -- 172.21.15.26:0/1866327690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3f830 msgr2=0x55d4f4e42c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14c77fe640 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3f830 0x55d4f4e42c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14c77fe640 1 -- 172.21.15.26:0/1866327690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3ee40 msgr2=0x55d4f4e426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14c77fe640 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14c77fe640 1 -- 172.21.15.26:0/1866327690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4f4e4f120 con 0x55d4f4e402d0 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14c7fff640 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3f830 0x55d4f4e42c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14c77fe640 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 0x55d4f4e4eb30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f14b80043b0 tx=0x7f14b8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b235ed421c021f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.684+0000 7f14a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14a77fe640 1 -- 172.21.15.26:0/1866327690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8003950 con 0x55d4f4e402d0 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14d160e740 1 -- 172.21.15.26:0/1866327690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b8004450 con 0x55d4f4e402d0 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14a77fe640 1 -- 172.21.15.26:0/1866327690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f14b8003af0 con 0x55d4f4e402d0 2024-04-21T10:30:49.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14a77fe640 1 -- 172.21.15.26:0/1866327690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f14b8005020 con 0x55d4f4e402d0 2024-04-21T10:30:49.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14d160e740 10 client.15246 fetch_fsmap learned FSMap version 20 2024-04-21T10:30:49.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14d160e740 1 -- 172.21.15.26:0/1866327690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d4f4e50420 con 0x55d4f4e402d0 2024-04-21T10:30:49.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14a77fe640 1 -- 172.21.15.26:0/1866327690 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b8004450 con 0x55d4f4e402d0 2024-04-21T10:30:49.689 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:49 smithi026 ceph-mon[27965]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T10:30:49.689 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:49 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1038552722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:30:49.689 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:49 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3948048150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:49.689 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:49 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/456823547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14a77fe640 1 -- 172.21.15.26:0/1866327690 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f14b8004b50 con 0x55d4f4e402d0 2024-04-21T10:30:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14d160e740 10 client.15246 fetch_fsmap finished waiting for FSMap version 20 2024-04-21T10:30:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14d160e740 10 client.15246 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T10:30:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14d160e740 20 client.15246 populate_metadata read hostname 'smithi026' 2024-04-21T10:30:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14d160e740 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x55d4f4ec0090 0x55d4f4ebcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14d160e740 4 client.15246 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T10:30:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.685+0000 7f14d160e740 1 -- 172.21.15.26:0/1866327690 --> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] -- command(tid 0: {"prefix": "status"}) -- 0x55d4f4dab3a0 con 0x55d4f4ec0090 2024-04-21T10:30:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.686+0000 7f14c7fff640 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x55d4f4ec0090 0x55d4f4ebcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.686+0000 7f14c7fff640 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x55d4f4ec0090 0x55d4f4ebcba0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=325bfc15a09695ab server_cookie=2a26a5f47618c4b6 in_seq=0 out_seq=0 2024-04-21T10:30:49.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.686+0000 7f14a77fe640 10 client.15246 ms_handle_connect on v2:172.21.15.169:6838/2428626465 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.686+0000 7f14a77fe640 1 -- 172.21.15.26:0/1866327690 <== mds.1 v2:172.21.15.169:6838/2428626465 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55d4f4dab3a0 con 0x55d4f4ec0090 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.686+0000 7f14a77fe640 10 client.15246 handle_command_reply: tid=0 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.686+0000 7f14d160e740 1 client.15246 shutdown 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14d160e740 1 -- 172.21.15.26:0/1866327690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 msgr2=0x55d4f4e4eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14d160e740 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 0x55d4f4e4eb30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f14b80043b0 tx=0x7f14b8014040 comp rx=0 tx=0).stop 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14d160e740 1 -- 172.21.15.26:0/1866327690 shutdown_connections 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14d160e740 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x55d4f4ec0090 0x55d4f4ebcba0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14d160e740 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d4f4e402d0 0x55d4f4e4eb30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14d160e740 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d4f4e3f830 0x55d4f4e42c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14d160e740 1 --2- 172.21.15.26:0/1866327690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d4f4e3ee40 0x55d4f4e426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14a77fe640 0 client.15246 ms_handle_reset on v2:172.21.15.169:6838/2428626465 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14d160e740 1 -- 172.21.15.26:0/1866327690 >> 172.21.15.26:0/1866327690 conn(0x55d4f4e3af40 msgr2=0x55d4f4e38170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14d160e740 1 -- 172.21.15.26:0/1866327690 shutdown_connections 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14d160e740 1 -- 172.21.15.26:0/1866327690 wait complete. 2024-04-21T10:30:49.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14d160e740 20 client.15246 trim_cache size 0 max 16384 2024-04-21T10:30:49.693 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:30:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:30:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-04-21T10:30:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24488, 2024-04-21T10:30:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-21T10:30:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-21T10:30:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-21T10:30:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 105.891492899, 2024-04-21T10:30:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 20, 2024-04-21T10:30:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 89, 2024-04-21T10:30:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 87, 2024-04-21T10:30:49.693 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 113.361526887 2024-04-21T10:30:49.694 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:30:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14ce94a640 1 -- 172.21.15.26:0/766364482 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f14b40a3bb0 msgr2=0x7f14b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14ce94a640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f14b40a3bb0 0x7f14b40a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f14c00bb490 tx=0x7f14c0089110 comp rx=0 tx=0).stop 2024-04-21T10:30:49.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14ce94a640 1 -- 172.21.15.26:0/766364482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14c8102530 msgr2=0x7f14c81a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14ce94a640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14c8102530 0x7f14c81a1890 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f14c002c700 tx=0x7f14c0034760 comp rx=0 tx=0).stop 2024-04-21T10:30:49.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14ce94a640 1 -- 172.21.15.26:0/766364482 shutdown_connections 2024-04-21T10:30:49.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14ce94a640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f14b40a3bb0 0x7f14b40a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14ce94a640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14c8102530 0x7f14c81a1890 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14ce94a640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c81009e0 0x7f14c819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14ce94a640 1 --2- 172.21.15.26:0/766364482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c8100020 0x7f14c819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14ce94a640 1 -- 172.21.15.26:0/766364482 >> 172.21.15.26:0/766364482 conn(0x7f14c80fbb80 msgr2=0x7f14c80fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14ce94a640 1 -- 172.21.15.26:0/766364482 shutdown_connections 2024-04-21T10:30:49.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.687+0000 7f14ce94a640 1 -- 172.21.15.26:0/766364482 wait complete. 2024-04-21T10:30:49.698 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T10:30:49.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:49 smithi169 ceph-mon[31469]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T10:30:49.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:49 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1038552722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:30:49.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:49 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3948048150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:49.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:49 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/456823547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:49.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:49 smithi190 ceph-mon[31423]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T10:30:49.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:49 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1038552722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:30:49.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:49 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3948048150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:30:49.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:49 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/456823547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:30:49.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.801+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/1706901309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54101fb0 msgr2=0x7fcc541023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.801+0000 7fcc5c7ca640 1 --2- 172.21.15.26:0/1706901309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54101fb0 0x7fcc541023d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009900 tx=0x7fcc4c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.801+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/1706901309 shutdown_connections 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.801+0000 7fcc5c7ca640 1 --2- 172.21.15.26:0/1706901309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc54103b60 0x7fcc5410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.801+0000 7fcc5c7ca640 1 --2- 172.21.15.26:0/1706901309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc541031a0 0x7fcc54103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.801+0000 7fcc5c7ca640 1 --2- 172.21.15.26:0/1706901309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54101fb0 0x7fcc541023d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.801+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/1706901309 >> 172.21.15.26:0/1706901309 conn(0x7fcc540fd520 msgr2=0x7fcc540ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.801+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/1706901309 shutdown_connections 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.801+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/1706901309 wait complete. 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc5c7ca640 1 Processor -- start 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc5c7ca640 1 -- start start 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc5c7ca640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc54101fb0 0x7fcc54075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc5c7ca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc541031a0 0x7fcc54074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc5c7ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc54103b60 0x7fcc54074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc5c7ca640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5410dba0 con 0x7fcc541031a0 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc5c7ca640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5410da20 con 0x7fcc54101fb0 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc5c7ca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5410dd20 con 0x7fcc54103b60 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc59d65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc541031a0 0x7fcc54074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc59d65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc541031a0 0x7fcc54074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58390/0 (socket says 172.21.15.26:58390) 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc59d65640 1 -- 172.21.15.26:0/2690435623 learned_addr learned my addr 172.21.15.26:0/2690435623 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc5ad67640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc54103b60 0x7fcc54074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc5a566640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc54101fb0 0x7fcc54075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc59d65640 1 -- 172.21.15.26:0/2690435623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc54103b60 msgr2=0x7fcc54074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc59d65640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc54103b60 0x7fcc54074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc59d65640 1 -- 172.21.15.26:0/2690435623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc54101fb0 msgr2=0x7fcc54075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc59d65640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc54101fb0 0x7fcc54075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc59d65640 1 -- 172.21.15.26:0/2690435623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc54075110 con 0x7fcc541031a0 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc5ad67640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc54103b60 0x7fcc54074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:49.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc5a566640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc54101fb0 0x7fcc54075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc59d65640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc541031a0 0x7fcc54074340 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400b4a0 tx=0x7fcc4400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.802+0000 7fcc4b7fe640 1 -- 172.21.15.26:0/2690435623 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44014070 con 0x7fcc541031a0 2024-04-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.803+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/2690435623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc541a2390 con 0x7fcc541031a0 2024-04-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.803+0000 7fcc4b7fe640 1 -- 172.21.15.26:0/2690435623 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc44010ac0 con 0x7fcc541031a0 2024-04-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.803+0000 7fcc4b7fe640 1 -- 172.21.15.26:0/2690435623 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44017c30 con 0x7fcc541031a0 2024-04-21T10:30:49.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.803+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/2690435623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc541a2960 con 0x7fcc541031a0 2024-04-21T10:30:49.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.805+0000 7fcc4b7fe640 1 -- 172.21.15.26:0/2690435623 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcc44017070 con 0x7fcc541031a0 2024-04-21T10:30:49.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.805+0000 7fcc5f3b6740 1 Processor -- start 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.805+0000 7fcc5f3b6740 1 -- start start 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.805+0000 7fcc5f3b6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 0x564ea291e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.805+0000 7fcc5f3b6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291e830 0x564ea291ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.805+0000 7fcc5f3b6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291f2d0 0x564ea2925eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.805+0000 7fcc5f3b6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564ea2882b10 con 0x564ea291de40 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.805+0000 7fcc5f3b6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ea2820240 con 0x564ea291e830 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.805+0000 7fcc5f3b6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564ea287ead0 con 0x564ea291f2d0 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc4b7fe640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcc400a3bb0 0x7fcc400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5ad67640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 0x564ea291e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5ad67640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 0x564ea291e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58394/0 (socket says 172.21.15.26:58394) 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5ad67640 1 -- 172.21.15.26:0/2730539526 learned_addr learned my addr 172.21.15.26:0/2730539526 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5a566640 1 --2- 172.21.15.26:0/2730539526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291e830 0x564ea291ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc59d65640 1 --2- 172.21.15.26:0/2730539526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291f2d0 0x564ea2925eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5ad67640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcc400a3bb0 0x7fcc400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc4b7fe640 1 -- 172.21.15.26:0/2690435623 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fcc440cfc00 con 0x7fcc541031a0 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5ad67640 1 -- 172.21.15.26:0/2730539526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291f2d0 msgr2=0x564ea2925eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5ad67640 1 --2- 172.21.15.26:0/2730539526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291f2d0 0x564ea2925eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5ad67640 1 -- 172.21.15.26:0/2730539526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291e830 msgr2=0x564ea291ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5ad67640 1 --2- 172.21.15.26:0/2730539526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291e830 0x564ea291ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5ad67640 1 -- 172.21.15.26:0/2730539526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea25952d0 con 0x564ea291de40 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5a566640 1 --2- 172.21.15.26:0/2730539526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291e830 0x564ea291ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc59d65640 1 --2- 172.21.15.26:0/2730539526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291f2d0 0x564ea2925eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5ad67640 1 --2- 172.21.15.26:0/2730539526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 0x564ea291e260 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcc50014a90 tx=0x7fcc50014f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68de3bd40914a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc48ff9640 1 -- 172.21.15.26:0/2730539526 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50014530 con 0x564ea291de40 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc48ff9640 1 -- 172.21.15.26:0/2730539526 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc50023070 con 0x564ea291de40 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc48ff9640 1 -- 172.21.15.26:0/2730539526 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000ccf0 con 0x564ea291de40 2024-04-21T10:30:49.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.806+0000 7fcc5ad67640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcc400a3bb0 0x7fcc400a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcc50022000 tx=0x7fcc50020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.807+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/2730539526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 msgr2=0x564ea291e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.807+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/2730539526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 0x564ea291e260 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcc50014a90 tx=0x7fcc50014f50 comp rx=0 tx=0).stop 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.807+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/2730539526 shutdown_connections 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.807+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/2730539526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291f2d0 0x564ea2925eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.807+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/2730539526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291e830 0x564ea291ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.807+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/2730539526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 0x564ea291e260 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.807+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/2730539526 >> 172.21.15.26:0/2730539526 conn(0x564ea2919f40 msgr2=0x564ea2918490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.807+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/2730539526 shutdown_connections 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.807+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/2730539526 wait complete. 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.807+0000 7fcc5f3b6740 1 -- start start 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.807+0000 7fcc5f3b6740 1 Processor -- start 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5f3b6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 0x564ea2927e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5f3b6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291e830 0x564ea29283c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5f3b6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291f2d0 0x564ea292f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5f3b6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564ea29264b0 con 0x564ea291de40 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5f3b6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ea280b190 con 0x564ea291f2d0 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5f3b6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564ea2926630 con 0x564ea291e830 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5f3b6740 10 client.0 _ll_register_callbacks cb 0x564ea28068b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5a566640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 0x564ea2927e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5a566640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 0x564ea2927e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58400/0 (socket says 172.21.15.26:58400) 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5a566640 1 -- 172.21.15.26:0/43521029 learned_addr learned my addr 172.21.15.26:0/43521029 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc59d65640 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291e830 0x564ea29283c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5a566640 1 -- 172.21.15.26:0/43521029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291e830 msgr2=0x564ea29283c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5a566640 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291e830 0x564ea29283c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5ad67640 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291f2d0 0x564ea292f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5a566640 1 -- 172.21.15.26:0/43521029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291f2d0 msgr2=0x564ea292f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5a566640 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291f2d0 0x564ea292f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5a566640 1 -- 172.21.15.26:0/43521029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea265f6b0 con 0x564ea291de40 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc5a566640 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 0x564ea2927e80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c000c00 tx=0x7fcc4c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4881d15763aaf04 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc3dffb640 1 -- 172.21.15.26:0/43521029 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c005030 con 0x564ea291de40 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.808+0000 7fcc3dffb640 1 -- 172.21.15.26:0/43521029 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fcc4c0051d0 con 0x564ea291de40 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/43521029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc50023070 con 0x564ea291de40 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc3dffb640 1 -- 172.21.15.26:0/43521029 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcc4c0053d0 con 0x564ea291de40 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc5f3b6740 10 client.15258 fetch_fsmap learned FSMap version 20 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/43521029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564ea29327e0 con 0x564ea291de40 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc3dffb640 1 -- 172.21.15.26:0/43521029 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc50023070 con 0x564ea291de40 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc3dffb640 1 -- 172.21.15.26:0/43521029 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fcc4c0039e0 con 0x564ea291de40 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc5f3b6740 10 client.15258 fetch_fsmap finished waiting for FSMap version 20 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc5f3b6740 10 client.15258 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc5f3b6740 20 client.15258 populate_metadata read hostname 'smithi026' 2024-04-21T10:30:49.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x564ea2934ef0 0x564ea2936360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc5f3b6740 4 client.15258 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/43521029 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ea2890530 con 0x564ea2934ef0 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.809+0000 7fcc59d65640 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x564ea2934ef0 0x564ea2936360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.810+0000 7fcc59d65640 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x564ea2934ef0 0x564ea2936360 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61aae83b303efe7b server_cookie=56ab53d5b96e1d5c in_seq=0 out_seq=0 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.810+0000 7fcc3dffb640 10 client.15258 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.810+0000 7fcc3dffb640 1 -- 172.21.15.26:0/43521029 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ea2890530 con 0x564ea2934ef0 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.810+0000 7fcc3dffb640 10 client.15258 handle_command_reply: tid=0 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.810+0000 7fcc5f3b6740 1 client.15258 shutdown 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.811+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/43521029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 msgr2=0x564ea2927e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.811+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 0x564ea2927e80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c000c00 tx=0x7fcc4c004270 comp rx=0 tx=0).stop 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.811+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/43521029 shutdown_connections 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.811+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x564ea2934ef0 0x564ea2936360 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.811+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291f2d0 0x564ea292f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.811+0000 7fcc3dffb640 0 client.15258 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:30:49.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.811+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291e830 0x564ea29283c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.811+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/43521029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291de40 0x564ea2927e80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.811+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/43521029 >> 172.21.15.26:0/43521029 conn(0x564ea2919f40 msgr2=0x564ea2917a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.811+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/43521029 shutdown_connections 2024-04-21T10:30:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.811+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/43521029 wait complete. 2024-04-21T10:30:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.811+0000 7fcc5f3b6740 20 client.15258 trim_cache size 0 max 16384 2024-04-21T10:30:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc5f3b6740 1 Processor -- start 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc5f3b6740 1 -- start start 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc5f3b6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291de40 0x564ea2917170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc5f3b6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 0x564ea292f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc5f3b6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291f2d0 0x564ea29176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc5f3b6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564ea29346a0 con 0x564ea291e830 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc5f3b6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ea2934d70 con 0x564ea291de40 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc5f3b6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564ea2935440 con 0x564ea291f2d0 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc59d65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 0x564ea292f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc59d65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 0x564ea292f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58406/0 (socket says 172.21.15.26:58406) 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc59d65640 1 -- 172.21.15.26:0/680613157 learned_addr learned my addr 172.21.15.26:0/680613157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc5ad67640 1 --2- 172.21.15.26:0/680613157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291f2d0 0x564ea29176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc5a566640 1 --2- 172.21.15.26:0/680613157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291de40 0x564ea2917170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc59d65640 1 -- 172.21.15.26:0/680613157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291f2d0 msgr2=0x564ea29176b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc59d65640 1 --2- 172.21.15.26:0/680613157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291f2d0 0x564ea29176b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc59d65640 1 -- 172.21.15.26:0/680613157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291de40 msgr2=0x564ea2917170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc59d65640 1 --2- 172.21.15.26:0/680613157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291de40 0x564ea2917170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc59d65640 1 -- 172.21.15.26:0/680613157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea2919230 con 0x564ea291e830 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc5a566640 1 --2- 172.21.15.26:0/680613157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291de40 0x564ea2917170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.829+0000 7fcc5ad67640 1 --2- 172.21.15.26:0/680613157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291f2d0 0x564ea29176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc59d65640 1 --2- 172.21.15.26:0/680613157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 0x564ea292f3c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcc440988a0 tx=0x7fcc44051040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181d6ccb872defb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc3ffff640 1 -- 172.21.15.26:0/680613157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc440b4070 con 0x564ea291e830 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc3ffff640 1 -- 172.21.15.26:0/680613157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc440a4b40 con 0x564ea291e830 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc3ffff640 1 -- 172.21.15.26:0/680613157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4401e040 con 0x564ea291e830 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/680613157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 msgr2=0x564ea292f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/680613157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 0x564ea292f3c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcc440988a0 tx=0x7fcc44051040 comp rx=0 tx=0).stop 2024-04-21T10:30:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/680613157 shutdown_connections 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/680613157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291f2d0 0x564ea29176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/680613157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 0x564ea292f3c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/680613157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291de40 0x564ea2917170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/680613157 >> 172.21.15.26:0/680613157 conn(0x564ea2919f40 msgr2=0x564ea2925df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/680613157 shutdown_connections 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.830+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/680613157 wait complete. 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc5f3b6740 1 -- start start 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc5f3b6740 1 Processor -- start 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc5f3b6740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291de40 0x564ea292af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc5f3b6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 0x564ea292b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc5f3b6740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291f2d0 0x564ea292e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc5f3b6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564ea29361e0 con 0x564ea291e830 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc5f3b6740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ea2935b10 con 0x564ea291f2d0 2024-04-21T10:30:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc5f3b6740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x564ea29368b0 con 0x564ea291de40 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc5f3b6740 10 client.0 _ll_register_callbacks cb 0x7fcc4c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc59d65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 0x564ea292b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc59d65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 0x564ea292b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58412/0 (socket says 172.21.15.26:58412) 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc59d65640 1 -- 172.21.15.26:0/540160444 learned_addr learned my addr 172.21.15.26:0/540160444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc5ad67640 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291f2d0 0x564ea292e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc5a566640 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291de40 0x564ea292af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc59d65640 1 -- 172.21.15.26:0/540160444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291de40 msgr2=0x564ea292af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc59d65640 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291de40 0x564ea292af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc59d65640 1 -- 172.21.15.26:0/540160444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291f2d0 msgr2=0x564ea292e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc59d65640 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291f2d0 0x564ea292e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc59d65640 1 -- 172.21.15.26:0/540160444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea292eaa0 con 0x564ea291e830 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc59d65640 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 0x564ea292b490 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcc440982e0 tx=0x7fcc44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b271c309ccfb82 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc3dffb640 1 -- 172.21.15.26:0/540160444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc440ba060 con 0x564ea291e830 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc3dffb640 1 -- 172.21.15.26:0/540160444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fcc44098540 con 0x564ea291e830 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.831+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/540160444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc440a4b40 con 0x564ea291e830 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.832+0000 7fcc3dffb640 1 -- 172.21.15.26:0/540160444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcc440b4070 con 0x564ea291e830 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.832+0000 7fcc5f3b6740 10 client.15264 fetch_fsmap learned FSMap version 20 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.832+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/540160444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564ea299b270 con 0x564ea291e830 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.832+0000 7fcc3dffb640 1 -- 172.21.15.26:0/540160444 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc440a4b40 con 0x564ea291e830 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.832+0000 7fcc3dffb640 1 -- 172.21.15.26:0/540160444 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7fcc440a4740 con 0x564ea291e830 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.832+0000 7fcc5f3b6740 10 client.15264 fetch_fsmap finished waiting for FSMap version 20 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.832+0000 7fcc5f3b6740 10 client.15264 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.832+0000 7fcc5f3b6740 20 client.15264 populate_metadata read hostname 'smithi026' 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.832+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x564ea299f990 0x564ea299e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.832+0000 7fcc5f3b6740 4 client.15264 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T10:30:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.832+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/540160444 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "status"}) -- 0x564ea288a3a0 con 0x564ea299f990 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.832+0000 7fcc5a566640 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x564ea299f990 0x564ea299e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc3dffb640 10 client.15264 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5a566640 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x564ea299f990 0x564ea299e6a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1912062df85faeaa server_cookie=4740ef52d33f80a in_seq=0 out_seq=0 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc3dffb640 1 -- 172.21.15.26:0/540160444 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x564ea288a3a0 con 0x564ea299f990 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc3dffb640 10 client.15264 handle_command_reply: tid=0 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5f3b6740 1 client.15264 shutdown 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/540160444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 msgr2=0x564ea292b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 0x564ea292b490 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcc440982e0 tx=0x7fcc44000bc0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/540160444 shutdown_connections 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x564ea299f990 0x564ea299e6a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ea291f2d0 0x564ea292e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ea291e830 0x564ea292b490 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5f3b6740 1 --2- 172.21.15.26:0/540160444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564ea291de40 0x564ea292af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc3dffb640 0 client.15264 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:30:49.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/540160444 >> 172.21.15.26:0/540160444 conn(0x564ea2919f40 msgr2=0x564ea2917170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/540160444 shutdown_connections 2024-04-21T10:30:49.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5f3b6740 1 -- 172.21.15.26:0/540160444 wait complete. 2024-04-21T10:30:49.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.833+0000 7fcc5f3b6740 20 client.15264 trim_cache size 0 max 16384 2024-04-21T10:30:49.840 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:30:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:30:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-21T10:30:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24496, 2024-04-21T10:30:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-21T10:30:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-21T10:30:49.840 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-21T10:30:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 107.921264557, 2024-04-21T10:30:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 20, 2024-04-21T10:30:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 89, 2024-04-21T10:30:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 87, 2024-04-21T10:30:49.841 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 121.869418104 2024-04-21T10:30:49.841 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:30:49.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.834+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/2690435623 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcc400a3bb0 msgr2=0x7fcc400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.834+0000 7fcc5c7ca640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcc400a3bb0 0x7fcc400a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcc50022000 tx=0x7fcc50020000 comp rx=0 tx=0).stop 2024-04-21T10:30:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.834+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/2690435623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc541031a0 msgr2=0x7fcc54074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.834+0000 7fcc5c7ca640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc541031a0 0x7fcc54074340 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400b4a0 tx=0x7fcc4400b960 comp rx=0 tx=0).stop 2024-04-21T10:30:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.834+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/2690435623 shutdown_connections 2024-04-21T10:30:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.834+0000 7fcc5c7ca640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fcc400a3bb0 0x7fcc400a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.834+0000 7fcc5c7ca640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc54103b60 0x7fcc54074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.834+0000 7fcc5c7ca640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc541031a0 0x7fcc54074340 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.834+0000 7fcc5c7ca640 1 --2- 172.21.15.26:0/2690435623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc54101fb0 0x7fcc54075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.834+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/2690435623 >> 172.21.15.26:0/2690435623 conn(0x7fcc540fd520 msgr2=0x7fcc54106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.834+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/2690435623 shutdown_connections 2024-04-21T10:30:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.834+0000 7fcc5c7ca640 1 -- 172.21.15.26:0/2690435623 wait complete. 2024-04-21T10:30:49.844 INFO:teuthology.run_tasks:Running task sequential... 2024-04-21T10:30:49.854 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-21T10:30:49.854 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-21T10:30:49.855 INFO:teuthology.task.print:Enabling mgr modules 2024-04-21T10:30:49.855 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-21T10:30:49.855 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T10:30:49.855 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-04-21T10:30:49.856 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.957+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2275083529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb01020e0 msgr2=0x7fdbb0102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.957+0000 7fdbb69a4640 1 --2- 172.21.15.26:0/2275083529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb01020e0 0x7fdbb0102560 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009900 tx=0x7fdba002bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.957+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2275083529 shutdown_connections 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.957+0000 7fdbb69a4640 1 --2- 172.21.15.26:0/2275083529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb0102aa0 0x7fdbb01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.957+0000 7fdbb69a4640 1 --2- 172.21.15.26:0/2275083529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb01020e0 0x7fdbb0102560 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.957+0000 7fdbb69a4640 1 --2- 172.21.15.26:0/2275083529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb0100f10 0x7fdbb0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.957+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2275083529 >> 172.21.15.26:0/2275083529 conn(0x7fdbb00fc4d0 msgr2=0x7fdbb00fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.957+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2275083529 shutdown_connections 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.957+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2275083529 wait complete. 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.957+0000 7fdbb69a4640 1 Processor -- start 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.957+0000 7fdbb69a4640 1 -- start start 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb69a4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb0100f10 0x7fdbb0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb69a4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb01020e0 0x7fdbb0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb69a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0102aa0 0x7fdbb019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb69a4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010ca90 con 0x7fdbb0102aa0 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb69a4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010c910 con 0x7fdbb01020e0 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb69a4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010cc10 con 0x7fdbb0100f10 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb4f41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0102aa0 0x7fdbb019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb4f41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0102aa0 0x7fdbb019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58442/0 (socket says 172.21.15.26:58442) 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb4f41640 1 -- 172.21.15.26:0/2994891439 learned_addr learned my addr 172.21.15.26:0/2994891439 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbaf7fe640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb01020e0 0x7fdbb0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbaffff640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb0100f10 0x7fdbb0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb4f41640 1 -- 172.21.15.26:0/2994891439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb0100f10 msgr2=0x7fdbb0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb4f41640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb0100f10 0x7fdbb0198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb4f41640 1 -- 172.21.15.26:0/2994891439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb01020e0 msgr2=0x7fdbb0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb4f41640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb01020e0 0x7fdbb0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb4f41640 1 -- 172.21.15.26:0/2994891439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb019fde0 con 0x7fdbb0102aa0 2024-04-21T10:30:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbaffff640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb0100f10 0x7fdbb0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:30:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb4f41640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0102aa0 0x7fdbb019f770 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdba800b970 tx=0x7fdba800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbad7fa640 1 -- 172.21.15.26:0/2994891439 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8019070 con 0x7fdbb0102aa0 2024-04-21T10:30:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbad7fa640 1 -- 172.21.15.26:0/2994891439 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba8014440 con 0x7fdbb0102aa0 2024-04-21T10:30:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2994891439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb01a00d0 con 0x7fdbb0102aa0 2024-04-21T10:30:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.958+0000 7fdbad7fa640 1 -- 172.21.15.26:0/2994891439 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80075d0 con 0x7fdbb0102aa0 2024-04-21T10:30:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.959+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2994891439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb01a0650 con 0x7fdbb0102aa0 2024-04-21T10:30:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.959+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2994891439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb01a0910 con 0x7fdbb0102aa0 2024-04-21T10:30:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.961+0000 7fdbad7fa640 1 -- 172.21.15.26:0/2994891439 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdba801c070 con 0x7fdbb0102aa0 2024-04-21T10:30:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.961+0000 7fdbad7fa640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fdb9c0a3c50 0x7fdb9c0a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.961+0000 7fdbaffff640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fdb9c0a3c50 0x7fdb9c0a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.961+0000 7fdbad7fa640 1 -- 172.21.15.26:0/2994891439 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fdba80cff80 con 0x7fdbb0102aa0 2024-04-21T10:30:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.962+0000 7fdbaffff640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fdb9c0a3c50 0x7fdb9c0a6110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0100c30 tx=0x7fdb98005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:49.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:49.965+0000 7fdbad7fa640 1 -- 172.21.15.26:0/2994891439 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdba809c950 con 0x7fdbb0102aa0 2024-04-21T10:30:50.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.203+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2994891439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fdbb01a0ae0 con 0x7fdbb0102aa0 2024-04-21T10:30:50.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.450+0000 7fdbad7fa640 1 -- 172.21.15.26:0/2994891439 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fdba809c250 con 0x7fdbb0102aa0 2024-04-21T10:30:50.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.453+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2994891439 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fdb9c0a3c50 msgr2=0x7fdb9c0a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:50.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.453+0000 7fdbb69a4640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fdb9c0a3c50 0x7fdb9c0a6110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0100c30 tx=0x7fdb98005e70 comp rx=0 tx=0).stop 2024-04-21T10:30:50.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.453+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2994891439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0102aa0 msgr2=0x7fdbb019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:50.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.453+0000 7fdbb69a4640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0102aa0 0x7fdbb019f770 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdba800b970 tx=0x7fdba800be30 comp rx=0 tx=0).stop 2024-04-21T10:30:50.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.453+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2994891439 shutdown_connections 2024-04-21T10:30:50.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.453+0000 7fdbb69a4640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fdb9c0a3c50 0x7fdb9c0a6110 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.453+0000 7fdbb69a4640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0102aa0 0x7fdbb019f770 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.453+0000 7fdbb69a4640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb01020e0 0x7fdbb0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.453+0000 7fdbb69a4640 1 --2- 172.21.15.26:0/2994891439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbb0100f10 0x7fdbb0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.453+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2994891439 >> 172.21.15.26:0/2994891439 conn(0x7fdbb00fc4d0 msgr2=0x7fdbb006a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:50.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.453+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2994891439 shutdown_connections 2024-04-21T10:30:50.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.453+0000 7fdbb69a4640 1 -- 172.21.15.26:0/2994891439 wait complete. 2024-04-21T10:30:50.470 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.580+0000 7f3a02214640 1 -- 172.21.15.26:0/3093072924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39fc101200 msgr2=0x7f39fc101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.580+0000 7f3a02214640 1 --2- 172.21.15.26:0/3093072924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39fc101200 0x7f39fc101660 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009900 tx=0x7f39f002bcd0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.580+0000 7f3a02214640 1 -- 172.21.15.26:0/3093072924 shutdown_connections 2024-04-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.580+0000 7f3a02214640 1 --2- 172.21.15.26:0/3093072924 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39fc101ba0 0x7f39fc102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.580+0000 7f3a02214640 1 --2- 172.21.15.26:0/3093072924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39fc101200 0x7f39fc101660 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.580+0000 7f3a02214640 1 --2- 172.21.15.26:0/3093072924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39fc1008a0 0x7f39fc100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.580+0000 7f3a02214640 1 -- 172.21.15.26:0/3093072924 >> 172.21.15.26:0/3093072924 conn(0x7f39fc0fc420 msgr2=0x7f39fc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.580+0000 7f3a02214640 1 -- 172.21.15.26:0/3093072924 shutdown_connections 2024-04-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.580+0000 7f3a02214640 1 -- 172.21.15.26:0/3093072924 wait complete. 2024-04-21T10:30:50.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f3a02214640 1 Processor -- start 2024-04-21T10:30:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f3a02214640 1 -- start start 2024-04-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f3a02214640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39fc1008a0 0x7f39fc19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f3a02214640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39fc101200 0x7f39fc19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f3a02214640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39fc101ba0 0x7f39fc1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f3a02214640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc10fcf0 con 0x7f39fc1008a0 2024-04-21T10:30:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f3a02214640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc10fb70 con 0x7f39fc101200 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f3a02214640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc10fe70 con 0x7f39fc101ba0 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f39fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39fc1008a0 0x7f39fc19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f39fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39fc1008a0 0x7f39fc19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58482/0 (socket says 172.21.15.26:58482) 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f39fb7fe640 1 -- 172.21.15.26:0/1991424230 learned_addr learned my addr 172.21.15.26:0/1991424230 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f39faffd640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39fc101200 0x7f39fc19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.581+0000 7f39fbfff640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39fc101ba0 0x7f39fc1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.582+0000 7f39fb7fe640 1 -- 172.21.15.26:0/1991424230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39fc101ba0 msgr2=0x7f39fc1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.582+0000 7f39fb7fe640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39fc101ba0 0x7f39fc1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.582+0000 7f39fb7fe640 1 -- 172.21.15.26:0/1991424230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39fc101200 msgr2=0x7f39fc19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.582+0000 7f39fb7fe640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39fc101200 0x7f39fc19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.582+0000 7f39fb7fe640 1 -- 172.21.15.26:0/1991424230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc1a2000 con 0x7f39fc1008a0 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.582+0000 7f39faffd640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39fc101200 0x7f39fc19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.582+0000 7f39fb7fe640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39fc1008a0 0x7f39fc19a3e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00e9f0 tx=0x7f39ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.582+0000 7f39f8ff9640 1 -- 172.21.15.26:0/1991424230 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00cd80 con 0x7f39fc1008a0 2024-04-21T10:30:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.582+0000 7f3a02214640 1 -- 172.21.15.26:0/1991424230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc1a22f0 con 0x7f39fc1008a0 2024-04-21T10:30:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.582+0000 7f39f8ff9640 1 -- 172.21.15.26:0/1991424230 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39ec0108e0 con 0x7f39fc1008a0 2024-04-21T10:30:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.582+0000 7f39f8ff9640 1 -- 172.21.15.26:0/1991424230 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec018980 con 0x7f39fc1008a0 2024-04-21T10:30:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.582+0000 7f3a02214640 1 -- 172.21.15.26:0/1991424230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc1a2810 con 0x7f39fc1008a0 2024-04-21T10:30:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.583+0000 7f3a02214640 1 -- 172.21.15.26:0/1991424230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39fc10b5f0 con 0x7f39fc1008a0 2024-04-21T10:30:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.585+0000 7f39f8ff9640 1 -- 172.21.15.26:0/1991424230 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f39ec010430 con 0x7f39fc1008a0 2024-04-21T10:30:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.586+0000 7f39f8ff9640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f39e40a3ca0 0x7f39e40a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.586+0000 7f39faffd640 1 -- 172.21.15.26:0/1991424230 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f39e40a3ca0 msgr2=0x7f39e40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/72610606 2024-04-21T10:30:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.586+0000 7f39f8ff9640 1 -- 172.21.15.26:0/1991424230 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f39ec0d0ca0 con 0x7f39fc1008a0 2024-04-21T10:30:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.586+0000 7f39faffd640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f39e40a3ca0 0x7f39e40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T10:30:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.590+0000 7f39f8ff9640 1 -- 172.21.15.26:0/1991424230 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f39ec09cfc0 con 0x7f39fc1008a0 2024-04-21T10:30:50.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:50 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2994891439' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T10:30:50.777 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:50 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: ignoring --setuser ceph since I am not root 2024-04-21T10:30:50.777 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:50 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: ignoring --setgroup ceph since I am not root 2024-04-21T10:30:50.777 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:50 smithi190 ceph-mgr[33298]: -- 172.21.15.190:0/1956409231 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560deb9b65a0 con 0x560deb98ac00 2024-04-21T10:30:50.777 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:50 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:50.634+0000 7fc725fb7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T10:30:50.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.778+0000 7f3a02214640 1 -- 172.21.15.26:0/1991424230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f39fc02d5c0 con 0x7f39fc1008a0 2024-04-21T10:30:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.782+0000 7f39f8ff9640 1 -- 172.21.15.26:0/1991424230 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) ==== 138+0+0 (secure 0 0 0) 0x7f39ec0a12c0 con 0x7f39fc1008a0 2024-04-21T10:30:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.784+0000 7f3a02214640 1 -- 172.21.15.26:0/1991424230 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f39e40a3ca0 msgr2=0x7f39e40a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T10:30:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.784+0000 7f3a02214640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f39e40a3ca0 0x7f39e40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.784+0000 7f3a02214640 1 -- 172.21.15.26:0/1991424230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39fc1008a0 msgr2=0x7f39fc19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.784+0000 7f3a02214640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39fc1008a0 0x7f39fc19a3e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00e9f0 tx=0x7f39ec00eeb0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.784+0000 7f3a02214640 1 -- 172.21.15.26:0/1991424230 shutdown_connections 2024-04-21T10:30:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.784+0000 7f3a02214640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f39e40a3ca0 0x7f39e40a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.784+0000 7f3a02214640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39fc101ba0 0x7f39fc1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.784+0000 7f3a02214640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39fc101200 0x7f39fc19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.784+0000 7f3a02214640 1 --2- 172.21.15.26:0/1991424230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39fc1008a0 0x7f39fc19a3e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.784+0000 7f3a02214640 1 -- 172.21.15.26:0/1991424230 >> 172.21.15.26:0/1991424230 conn(0x7f39fc0fc420 msgr2=0x7f39fc0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.784+0000 7f3a02214640 1 -- 172.21.15.26:0/1991424230 shutdown_connections 2024-04-21T10:30:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.784+0000 7f3a02214640 1 -- 172.21.15.26:0/1991424230 wait complete. 2024-04-21T10:30:50.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:50 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2994891439' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T10:30:50.794 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:50 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: ignoring --setuser ceph since I am not root 2024-04-21T10:30:50.794 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:50 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: ignoring --setgroup ceph since I am not root 2024-04-21T10:30:50.795 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:50 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:50.643+0000 7f55324f6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T10:30:50.795 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-21T10:30:50.815 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:50 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2994891439' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T10:30:50.815 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: ignoring --setuser ceph since I am not root 2024-04-21T10:30:50.816 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: ignoring --setgroup ceph since I am not root 2024-04-21T10:30:50.816 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:50 smithi169 ceph-mgr[32864]: -- 172.21.15.169:0/2026676325 <== mon.2 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d983165a0 con 0x559d982ea800 2024-04-21T10:30:50.816 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:50.631+0000 7f693886d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T10:30:50.816 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:50.816+0000 7f693886d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:30:50.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.900+0000 7f983ca9a640 1 -- 172.21.15.26:0/3374673892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9838102a00 msgr2=0x7f9838109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:50.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.900+0000 7f983ca9a640 1 --2- 172.21.15.26:0/3374673892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9838102a00 0x7f9838109320 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9828009ca0 tx=0x7f982802c0f0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.900+0000 7f983ca9a640 1 -- 172.21.15.26:0/3374673892 shutdown_connections 2024-04-21T10:30:50.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.900+0000 7f983ca9a640 1 --2- 172.21.15.26:0/3374673892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9838102a00 0x7f9838109320 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.900+0000 7f983ca9a640 1 --2- 172.21.15.26:0/3374673892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9838102040 0x7f98381024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.900+0000 7f983ca9a640 1 --2- 172.21.15.26:0/3374673892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9838100e60 0x7f9838101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.900+0000 7f983ca9a640 1 -- 172.21.15.26:0/3374673892 >> 172.21.15.26:0/3374673892 conn(0x7f98380fc420 msgr2=0x7f98380fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:50.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.900+0000 7f983ca9a640 1 -- 172.21.15.26:0/3374673892 shutdown_connections 2024-04-21T10:30:50.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.900+0000 7f983ca9a640 1 -- 172.21.15.26:0/3374673892 wait complete. 2024-04-21T10:30:50.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f983ca9a640 1 Processor -- start 2024-04-21T10:30:50.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f983ca9a640 1 -- start start 2024-04-21T10:30:50.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f983ca9a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9838100e60 0x7f9838198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:50.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f983ca9a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9838102040 0x7f9838198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:50.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f983ca9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9838102a00 0x7f983819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:50.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f983ca9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f983810db30 con 0x7f9838102a00 2024-04-21T10:30:50.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f983ca9a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f983810d9b0 con 0x7f9838102040 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f983ca9a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f983810dcb0 con 0x7f9838100e60 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f9836d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9838102a00 0x7f983819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f9835d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9838102040 0x7f9838198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f9836d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9838102a00 0x7f983819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58506/0 (socket says 172.21.15.26:58506) 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f9835d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9838102040 0x7f9838198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.26:60400/0 (socket says 172.21.15.26:60400) 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f9836d9d640 1 -- 172.21.15.26:0/956500975 learned_addr learned my addr 172.21.15.26:0/956500975 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f983659c640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9838100e60 0x7f9838198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f9836d9d640 1 -- 172.21.15.26:0/956500975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9838100e60 msgr2=0x7f9838198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f9836d9d640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9838100e60 0x7f9838198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f9836d9d640 1 -- 172.21.15.26:0/956500975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9838102040 msgr2=0x7f9838198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f9836d9d640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9838102040 0x7f9838198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.901+0000 7f9836d9d640 1 -- 172.21.15.26:0/956500975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983819fd40 con 0x7f9838102a00 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.902+0000 7f9836d9d640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9838102a00 0x7f983819f6d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f982802c700 tx=0x7f98280346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:50.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.902+0000 7f982f7fe640 1 -- 172.21.15.26:0/956500975 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828041020 con 0x7f9838102a00 2024-04-21T10:30:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.902+0000 7f983ca9a640 1 -- 172.21.15.26:0/956500975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983819ffd0 con 0x7f9838102a00 2024-04-21T10:30:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.902+0000 7f982f7fe640 1 -- 172.21.15.26:0/956500975 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9828034ca0 con 0x7f9838102a00 2024-04-21T10:30:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.902+0000 7f982f7fe640 1 -- 172.21.15.26:0/956500975 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828039040 con 0x7f9838102a00 2024-04-21T10:30:50.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.902+0000 7f983ca9a640 1 -- 172.21.15.26:0/956500975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98381a0490 con 0x7f9838102a00 2024-04-21T10:30:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.903+0000 7f983ca9a640 1 -- 172.21.15.26:0/956500975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9838108200 con 0x7f9838102a00 2024-04-21T10:30:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.904+0000 7f982f7fe640 1 -- 172.21.15.26:0/956500975 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f9828044020 con 0x7f9838102a00 2024-04-21T10:30:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.905+0000 7f982f7fe640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f98240a3ca0 0x7f98240a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.905+0000 7f983659c640 1 -- 172.21.15.26:0/956500975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f98240a3ca0 msgr2=0x7f98240a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/72610606 2024-04-21T10:30:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.905+0000 7f982f7fe640 1 -- 172.21.15.26:0/956500975 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f98280f21e0 con 0x7f9838102a00 2024-04-21T10:30:50.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.905+0000 7f983659c640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f98240a3ca0 0x7f98240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T10:30:50.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:50.909+0000 7f982f7fe640 1 -- 172.21.15.26:0/956500975 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f982804b020 con 0x7f9838102a00 2024-04-21T10:30:51.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.100+0000 7f983ca9a640 1 -- 172.21.15.26:0/956500975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f983802d5c0 con 0x7f9838102a00 2024-04-21T10:30:51.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.104+0000 7f982f7fe640 1 -- 172.21.15.26:0/956500975 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) ==== 133+0+0 (secure 0 0 0) 0x7f98280beb50 con 0x7f9838102a00 2024-04-21T10:30:51.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983659c640 1 -- 172.21.15.26:0/956500975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f98240a3ca0 msgr2=0x7f98240a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/72610606 2024-04-21T10:30:51.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983659c640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f98240a3ca0 0x7f98240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T10:30:51.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983ca9a640 1 -- 172.21.15.26:0/956500975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f98240a3ca0 msgr2=0x7f98240a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T10:30:51.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983ca9a640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f98240a3ca0 0x7f98240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:51.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983ca9a640 1 -- 172.21.15.26:0/956500975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9838102a00 msgr2=0x7f983819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:51.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983ca9a640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9838102a00 0x7f983819f6d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f982802c700 tx=0x7f98280346d0 comp rx=0 tx=0).stop 2024-04-21T10:30:51.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983ca9a640 1 -- 172.21.15.26:0/956500975 shutdown_connections 2024-04-21T10:30:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983ca9a640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f98240a3ca0 0x7f98240a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983ca9a640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9838102a00 0x7f983819f6d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983ca9a640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9838102040 0x7f9838198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983ca9a640 1 --2- 172.21.15.26:0/956500975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9838100e60 0x7f9838198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983ca9a640 1 -- 172.21.15.26:0/956500975 >> 172.21.15.26:0/956500975 conn(0x7f98380fc420 msgr2=0x7f98381053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983ca9a640 1 -- 172.21.15.26:0/956500975 shutdown_connections 2024-04-21T10:30:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:51.106+0000 7f983ca9a640 1 -- 172.21.15.26:0/956500975 wait complete. 2024-04-21T10:30:51.116 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:50 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:50.837+0000 7f55324f6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:30:51.116 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:50 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:50.948+0000 7f55324f6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T10:30:51.117 INFO:teuthology.run_tasks:Running task kclient... 2024-04-21T10:30:51.143 INFO:tasks.kclient:Mounting kernel clients... 2024-04-21T10:30:51.143 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-21T10:30:51.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:30:51.143 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-21T10:30:51.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:30:51.159 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-04-21T10:30:51.198 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:50.918+0000 7f693886d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T10:30:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-04-21T10:30:51.226 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:50 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:50.816+0000 7fc725fb7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:30:51.227 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:50 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:50.919+0000 7fc725fb7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T10:30:51.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:30:51.228 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-21T10:30:51.228 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-21T10:30:51.228 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-21T10:30:51.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph fs ls 2024-04-21T10:30:51.368 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:51 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:51.277+0000 7f55324f6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T10:30:51.368 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:51 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:51.368+0000 7f55324f6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T10:30:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:30:51.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:51 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2994891439' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T10:30:51.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:51 smithi190 ceph-mon[31423]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-21T10:30:51.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:51 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1991424230' entity='client.admin' 2024-04-21T10:30:51.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:51 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/956500975' entity='client.admin' 2024-04-21T10:30:51.527 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:51 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:51.227+0000 7fc725fb7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T10:30:51.527 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:51 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:51.312+0000 7fc725fb7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T10:30:51.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:51 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2994891439' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T10:30:51.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:51 smithi026 ceph-mon[27965]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-21T10:30:51.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:51 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1991424230' entity='client.admin' 2024-04-21T10:30:51.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:51 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/956500975' entity='client.admin' 2024-04-21T10:30:51.698 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:51 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:51.227+0000 7f693886d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T10:30:51.698 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:51 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:51.311+0000 7f693886d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T10:30:51.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:51 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2994891439' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T10:30:51.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:51 smithi169 ceph-mon[31469]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-21T10:30:51.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:51 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1991424230' entity='client.admin' 2024-04-21T10:30:51.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:51 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/956500975' entity='client.admin' 2024-04-21T10:30:52.776 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:52 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:52.484+0000 7fc725fb7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T10:30:52.777 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:52 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:52.651+0000 7fc725fb7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T10:30:52.840 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:52 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:52.499+0000 7f693886d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T10:30:52.841 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:52 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:52.669+0000 7f693886d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T10:30:52.841 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:52 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:52.841+0000 7f693886d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T10:30:52.935 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:52 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:52.648+0000 7f55324f6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T10:30:52.935 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:52 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:52.837+0000 7f55324f6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T10:30:53.107 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:52 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:52.820+0000 7fc725fb7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T10:30:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.238+0000 7f78c1c24700 1 -- 172.21.15.26:0/521854641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc102d50 msgr2=0x7f78bc1065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.238+0000 7f78c1c24700 1 --2- 172.21.15.26:0/521854641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc102d50 0x7f78bc1065e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f78b00099c0 tx=0x7f78b0009cd0 comp rx=0 tx=0).stop 2024-04-21T10:30:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.239+0000 7f78c1c24700 1 -- 172.21.15.26:0/521854641 shutdown_connections 2024-04-21T10:30:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.239+0000 7f78c1c24700 1 --2- 172.21.15.26:0/521854641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78bc1023a0 0x7f78bc102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.239+0000 7f78c1c24700 1 --2- 172.21.15.26:0/521854641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78bc106cc0 0x7f78bc10e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.239+0000 7f78c1c24700 1 --2- 172.21.15.26:0/521854641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc102d50 0x7f78bc1065e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.239+0000 7f78c1c24700 1 -- 172.21.15.26:0/521854641 >> 172.21.15.26:0/521854641 conn(0x7f78bc077fb0 msgr2=0x7f78bc0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.239+0000 7f78c1c24700 1 -- 172.21.15.26:0/521854641 shutdown_connections 2024-04-21T10:30:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.239+0000 7f78c1c24700 1 -- 172.21.15.26:0/521854641 wait complete. 2024-04-21T10:30:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.239+0000 7f78c1c24700 1 Processor -- start 2024-04-21T10:30:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.239+0000 7f78c1c24700 1 -- start start 2024-04-21T10:30:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78c1c24700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc1023a0 0x7f78bc19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78c1c24700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78bc102d50 0x7f78bc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78c1c24700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78bc106cc0 0x7f78bc1a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78c1c24700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc111bc0 con 0x7f78bc1023a0 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78c1c24700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc111a40 con 0x7f78bc102d50 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78c1c24700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc111d40 con 0x7f78bc106cc0 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78bb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc1023a0 0x7f78bc19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78bb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc1023a0 0x7f78bc19a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58524/0 (socket says 172.21.15.26:58524) 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78bb7fe700 1 -- 172.21.15.26:0/1279310072 learned_addr learned my addr 172.21.15.26:0/1279310072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78baffd700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78bc102d50 0x7f78bc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78bbfff700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78bc106cc0 0x7f78bc1a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78bb7fe700 1 -- 172.21.15.26:0/1279310072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78bc106cc0 msgr2=0x7f78bc1a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78bc106cc0 0x7f78bc1a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78bb7fe700 1 -- 172.21.15.26:0/1279310072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78bc102d50 msgr2=0x7f78bc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78bc102d50 0x7f78bc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78bb7fe700 1 -- 172.21.15.26:0/1279310072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc1a5680 con 0x7f78bc1023a0 2024-04-21T10:30:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78baffd700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78bc102d50 0x7f78bc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:30:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.240+0000 7f78bb7fe700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc1023a0 0x7f78bc19a0c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00e9a0 tx=0x7f78ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.241+0000 7f78b8ff9700 1 -- 172.21.15.26:0/1279310072 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00ca10 con 0x7f78bc1023a0 2024-04-21T10:30:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.241+0000 7f78c1c24700 1 -- 172.21.15.26:0/1279310072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78bc1a5850 con 0x7f78bc1023a0 2024-04-21T10:30:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.241+0000 7f78b8ff9700 1 -- 172.21.15.26:0/1279310072 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78ac00cbb0 con 0x7f78bc1023a0 2024-04-21T10:30:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.241+0000 7f78b8ff9700 1 -- 172.21.15.26:0/1279310072 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac018680 con 0x7f78bc1023a0 2024-04-21T10:30:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.241+0000 7f78c1c24700 1 -- 172.21.15.26:0/1279310072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78bc1a5db0 con 0x7f78bc1023a0 2024-04-21T10:30:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.242+0000 7f78c1c24700 1 -- 172.21.15.26:0/1279310072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78bc069730 con 0x7f78bc1023a0 2024-04-21T10:30:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.243+0000 7f78b8ff9700 1 -- 172.21.15.26:0/1279310072 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f78ac010430 con 0x7f78bc1023a0 2024-04-21T10:30:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.244+0000 7f78b8ff9700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f78a40a3d10 0x7f78a40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.244+0000 7f78baffd700 1 -- 172.21.15.26:0/1279310072 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f78a40a3d10 msgr2=0x7f78a40a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/72610606 2024-04-21T10:30:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.244+0000 7f78b8ff9700 1 -- 172.21.15.26:0/1279310072 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f78ac0d0880 con 0x7f78bc1023a0 2024-04-21T10:30:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.244+0000 7f78baffd700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f78a40a3d10 0x7f78a40a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T10:30:53.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.247+0000 7f78b8ff9700 1 -- 172.21.15.26:0/1279310072 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78ac09d250 con 0x7f78bc1023a0 2024-04-21T10:30:53.247 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:53 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:53.019+0000 7f55324f6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T10:30:53.402 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:53.107+0000 7fc725fb7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T10:30:53.402 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:53.200+0000 7fc725fb7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T10:30:53.430 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:53 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:53.132+0000 7f693886d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T10:30:53.431 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:53 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:53.226+0000 7f693886d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T10:30:53.431 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:53 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:53.431+0000 7f693886d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T10:30:53.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.445+0000 7f78baffd700 1 -- 172.21.15.26:0/1279310072 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f78a40a3d10 msgr2=0x7f78a40a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/72610606 2024-04-21T10:30:53.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.445+0000 7f78baffd700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f78a40a3d10 0x7f78a40a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T10:30:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.507+0000 7f78c1c24700 1 -- 172.21.15.26:0/1279310072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f78bc103160 con 0x7f78bc1023a0 2024-04-21T10:30:53.507 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:53 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:53.330+0000 7f55324f6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T10:30:53.507 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:53 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:53.427+0000 7f55324f6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T10:30:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.508+0000 7f78b8ff9700 1 -- 172.21.15.26:0/1279310072 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f78ac09cb50 con 0x7f78bc1023a0 2024-04-21T10:30:53.508 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-21T10:30:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.510+0000 7f78c1c24700 1 -- 172.21.15.26:0/1279310072 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f78a40a3d10 msgr2=0x7f78a40a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T10:30:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.510+0000 7f78c1c24700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f78a40a3d10 0x7f78a40a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.510+0000 7f78c1c24700 1 -- 172.21.15.26:0/1279310072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc1023a0 msgr2=0x7f78bc19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.510+0000 7f78c1c24700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc1023a0 0x7f78bc19a0c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00e9a0 tx=0x7f78ac00ecb0 comp rx=0 tx=0).stop 2024-04-21T10:30:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.510+0000 7f78c1c24700 1 -- 172.21.15.26:0/1279310072 shutdown_connections 2024-04-21T10:30:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.510+0000 7f78c1c24700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f78a40a3d10 0x7f78a40a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.510+0000 7f78c1c24700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78bc106cc0 0x7f78bc1a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.510+0000 7f78c1c24700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc1023a0 0x7f78bc19a0c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.510+0000 7f78c1c24700 1 --2- 172.21.15.26:0/1279310072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78bc102d50 0x7f78bc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:53.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.510+0000 7f78c1c24700 1 -- 172.21.15.26:0/1279310072 >> 172.21.15.26:0/1279310072 conn(0x7f78bc077fb0 msgr2=0x7f78bc0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:53.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.510+0000 7f78c1c24700 1 -- 172.21.15.26:0/1279310072 shutdown_connections 2024-04-21T10:30:53.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:53.510+0000 7f78c1c24700 1 -- 172.21.15.26:0/1279310072 wait complete. 2024-04-21T10:30:53.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:53 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1279310072' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T10:30:53.698 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:53 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:53.623+0000 7f693886d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:30:53.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:53 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1279310072' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T10:30:53.776 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:53.403+0000 7fc725fb7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T10:30:53.777 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:53.596+0000 7fc725fb7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:30:53.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:53 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1279310072' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T10:30:53.796 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:53 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:53.644+0000 7f55324f6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T10:30:54.146 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:53 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:53.848+0000 7f55324f6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T10:30:54.176 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-21T10:30:54.176 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-21T10:30:54.176 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-04-21T10:30:54.177 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-21T10:30:54.177 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-21T10:30:54.177 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-21T10:30:54.177 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-21T10:30:54.177 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-21T10:30:54.177 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-21T10:30:54.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:30:54.177 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-04-21T10:30:54.193 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-21T10:30:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-21T10:30:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-21T10:30:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-21T10:30:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-21T10:30:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-21T10:30:54.193 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T10:30:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-04-21T10:30:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-04-21T10:30:54.193 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T10:30:54.193 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-04-21T10:30:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-04-21T10:30:54.194 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-21T10:30:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-04-21T10:30:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-04-21T10:30:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-21T10:30:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42079sec preferred_lft 42079sec 2024-04-21T10:30:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-21T10:30:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-21T10:30:54.194 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T10:30:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-04-21T10:30:54.194 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-04-21T10:30:54.195 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-21T10:30:54.195 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T10:30:54.195 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-21T10:30:54.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2024-04-21T10:30:54.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2024-04-21T10:30:54.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2024-04-21T10:30:54.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-21T10:30:54.195 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-21T10:30:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T10:30:54.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T10:30:54.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:30:54.388 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-21T10:30:54.415 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-04-21T10:30:54.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:30:54.417 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-04-21T10:30:54.489 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-04-21T10:30:54.489 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-21T10:30:54.490 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-21T10:30:54.490 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-21T10:30:54.490 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-21T10:30:54.490 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T10:30:54.490 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-21T10:30:54.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-21T10:30:54.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-21T10:30:54.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-21T10:30:54.491 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-21T10:30:54.526 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:54.195+0000 7fc725fb7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T10:30:54.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T10:30:54.645 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:54.225+0000 7f693886d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T10:30:54.685 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:54 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:54.497+0000 7f55324f6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T10:30:54.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T10:30:54.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:30:54.768 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-21T10:30:54.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:30:54.784 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-04-21T10:30:54.839 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T10:30:54.839 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-21T10:30:54.839 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-21T10:30:54.839 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-21T10:30:54.839 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-21T10:30:54.905 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:54.612+0000 7fc725fb7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T10:30:54.905 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:54.707+0000 7fc725fb7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T10:30:54.905 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:54.905+0000 7fc725fb7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T10:30:54.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T10:30:54.943 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:54.646+0000 7f693886d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T10:30:54.943 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:54.742+0000 7f693886d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T10:30:54.943 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:54.943+0000 7f693886d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T10:30:54.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T10:30:54.959 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-21T10:30:54.959 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T10:30:54.959 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-21T10:30:54.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-21T10:30:54.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-21T10:30:54.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-21T10:30:54.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-21T10:30:54.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-21T10:30:54.959 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-21T10:30:55.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T10:30:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T10:30:55.096 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T10:30:55.096 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-21T10:30:55.096 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-04-21T10:30:55.096 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2024-04-21T10:30:55.096 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-21T10:30:55.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T10:30:55.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T10:30:55.174 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T10:30:55.174 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-21T10:30:55.189 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-21T10:30:55.189 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T10:30:55.189 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-21T10:30:55.244 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-21T10:30:55.244 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-21T10:30:55.244 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/731d5979-300b-4e3c-938a-e127814d4722 2024-04-21T10:30:55.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:30:55.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/731d5979-300b-4e3c-938a-e127814d4722 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-21T10:30:55.287 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:54 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:54.957+0000 7f55324f6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T10:30:55.287 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:55.065+0000 7f55324f6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T10:30:55.287 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:55.287+0000 7f55324f6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T10:30:55.336 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-21T10:30:55.336 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-21T10:30:55.336 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-21T10:30:55.463 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-21T10:30:55.463 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-21T10:30:55.463 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-04-21T10:30:55.463 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.169:3300,172.21.15.190:3300" 2024-04-21T10:30:55.463 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:3300,172.21.15.169:3300,172.21.15.190:3300:/volumes/qa/sv_1/731d5979-300b-4e3c-938a-e127814d4722 2024-04-21T10:30:55.463 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=b77245c8-ffc8-11ee-bc93-c7b262605968" will pass to kernel 2024-04-21T10:30:55.464 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-21T10:30:55.464 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-04-21T10:30:55.464 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-21T10:30:55.464 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-04-21T10:30:55.464 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-21T10:30:55.466 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-21T10:30:55.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:30:55.466 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-21T10:30:55.493 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T10:30:55.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T10:30:55.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T10:30:55.593 INFO:teuthology.orchestra.run.smithi026.stdout:client24979 2024-04-21T10:30:55.593 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:30:55.593 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T10:30:55.609 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T10:30:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T10:30:55.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T10:30:55.707 INFO:teuthology.orchestra.run.smithi026.stdout:client24979 2024-04-21T10:30:55.708 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T10:30:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T10:30:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T10:30:55.759 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:55.427+0000 7fc725fb7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T10:30:55.759 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:55.520+0000 7fc725fb7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T10:30:55.766 INFO:teuthology.orchestra.run.smithi026.stdout:client24979 2024-04-21T10:30:55.766 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968.client24979/status') 2024-04-21T10:30:55.801 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:55 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:55.468+0000 7f693886d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T10:30:55.801 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:55 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:55.560+0000 7f693886d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T10:30:55.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T10:30:55.826 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.24979 (3)172.21.15.26:0/3096504957 2024-04-21T10:30:55.826 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2024-04-21T10:30:55.826 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2024-04-21T10:30:55.826 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2024-04-21T10:30:55.826 INFO:teuthology.orchestra.run.smithi026.stderr:71 bytes copied, 2.9231e-05 s, 2.4 MB/s 2024-04-21T10:30:55.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T10:30:55.834 INFO:teuthology.run_tasks:Running task exec... 2024-04-21T10:30:55.843 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T10:30:55.843 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-04-21T10:30:55.843 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.949+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1535548038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb834102a30 msgr2=0x7fb834104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.949+0000 7fb83a9b6640 1 --2- 172.21.15.26:0/1535548038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb834102a30 0x7fb834104e20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009900 tx=0x7fb81c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.949+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1535548038 shutdown_connections 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.949+0000 7fb83a9b6640 1 --2- 172.21.15.26:0/1535548038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb834107c90 0x7fb83410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.949+0000 7fb83a9b6640 1 --2- 172.21.15.26:0/1535548038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb834105360 0x7fb834107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.949+0000 7fb83a9b6640 1 --2- 172.21.15.26:0/1535548038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb834102a30 0x7fb834104e20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.949+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1535548038 >> 172.21.15.26:0/1535548038 conn(0x7fb8340fc420 msgr2=0x7fb8340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.949+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1535548038 shutdown_connections 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.949+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1535548038 wait complete. 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb83a9b6640 1 Processor -- start 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb83a9b6640 1 -- start start 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb83a9b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb834102a30 0x7fb834198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:55.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb83a9b6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb834105360 0x7fb834198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb83a9b6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb834107c90 0x7fb83419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb83a9b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb83410daf0 con 0x7fb834102a30 2024-04-21T10:30:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb83a9b6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb83410d970 con 0x7fb834107c90 2024-04-21T10:30:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb83a9b6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb83410dc70 con 0x7fb834105360 2024-04-21T10:30:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb833fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb834102a30 0x7fb834198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb833fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb834102a30 0x7fb834198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45968/0 (socket says 172.21.15.26:45968) 2024-04-21T10:30:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb833fff640 1 -- 172.21.15.26:0/1699227083 learned_addr learned my addr 172.21.15.26:0/1699227083 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb8337fe640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb834105360 0x7fb834198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:55.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb838f53640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb834107c90 0x7fb83419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb833fff640 1 -- 172.21.15.26:0/1699227083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb834105360 msgr2=0x7fb834198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb833fff640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb834105360 0x7fb834198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb833fff640 1 -- 172.21.15.26:0/1699227083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb834107c90 msgr2=0x7fb83419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb833fff640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb834107c90 0x7fb83419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.950+0000 7fb833fff640 1 -- 172.21.15.26:0/1699227083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83419fd50 con 0x7fb834102a30 2024-04-21T10:30:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.951+0000 7fb833fff640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb834102a30 0x7fb834198130 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb81c006010 tx=0x7fb81c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.951+0000 7fb8317fa640 1 -- 172.21.15.26:0/1699227083 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c043020 con 0x7fb834102a30 2024-04-21T10:30:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.951+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1699227083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83419ffe0 con 0x7fb834102a30 2024-04-21T10:30:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.951+0000 7fb8317fa640 1 -- 172.21.15.26:0/1699227083 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb81c0043d0 con 0x7fb834102a30 2024-04-21T10:30:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.951+0000 7fb8317fa640 1 -- 172.21.15.26:0/1699227083 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c003b80 con 0x7fb834102a30 2024-04-21T10:30:55.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.951+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1699227083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8341a04a0 con 0x7fb834102a30 2024-04-21T10:30:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.952+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1699227083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8340fdf10 con 0x7fb834102a30 2024-04-21T10:30:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.953+0000 7fb8317fa640 1 -- 172.21.15.26:0/1699227083 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fb81c046020 con 0x7fb834102a30 2024-04-21T10:30:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.954+0000 7fb8317fa640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb8200a3ca0 0x7fb8200a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.954+0000 7fb8337fe640 1 -- 172.21.15.26:0/1699227083 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb8200a3ca0 msgr2=0x7fb8200a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/72610606 2024-04-21T10:30:55.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.954+0000 7fb8317fa640 1 -- 172.21.15.26:0/1699227083 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7fb81c0f2d50 con 0x7fb834102a30 2024-04-21T10:30:55.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.954+0000 7fb8337fe640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb8200a3ca0 0x7fb8200a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T10:30:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:55.957+0000 7fb8317fa640 1 -- 172.21.15.26:0/1699227083 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb81c04d020 con 0x7fb834102a30 2024-04-21T10:30:56.026 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:55.760+0000 7fc725fb7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T10:30:56.026 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:55.891+0000 7fc725fb7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T10:30:56.026 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:55.986+0000 7fc725fb7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T10:30:56.145 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:55 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:55.802+0000 7f693886d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T10:30:56.145 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:55 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:55.934+0000 7f693886d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T10:30:56.145 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:56 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:56.030+0000 7f693886d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T10:30:56.145 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:56 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:56.145+0000 7f693886d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T10:30:56.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.155+0000 7fb8337fe640 1 -- 172.21.15.26:0/1699227083 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb8200a3ca0 msgr2=0x7fb8200a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/72610606 2024-04-21T10:30:56.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.155+0000 7fb8337fe640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb8200a3ca0 0x7fb8200a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T10:30:56.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.185+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1699227083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fb83402d5c0 con 0x7fb834102a30 2024-04-21T10:30:56.185 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:55.855+0000 7f55324f6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T10:30:56.185 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:55.959+0000 7f55324f6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T10:30:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.237+0000 7fb8317fa640 1 -- 172.21.15.26:0/1699227083 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v21) ==== 118+0+0 (secure 0 0 0) 0x7fb81c0bf6c0 con 0x7fb834102a30 2024-04-21T10:30:56.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.239+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1699227083 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb8200a3ca0 msgr2=0x7fb8200a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T10:30:56.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.239+0000 7fb83a9b6640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb8200a3ca0 0x7fb8200a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:56.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.239+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1699227083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb834102a30 msgr2=0x7fb834198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:56.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.239+0000 7fb83a9b6640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb834102a30 0x7fb834198130 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb81c006010 tx=0x7fb81c005e40 comp rx=0 tx=0).stop 2024-04-21T10:30:56.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.239+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1699227083 shutdown_connections 2024-04-21T10:30:56.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.239+0000 7fb83a9b6640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7fb8200a3ca0 0x7fb8200a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:56.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.239+0000 7fb83a9b6640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb834107c90 0x7fb83419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:56.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.239+0000 7fb83a9b6640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb834105360 0x7fb834198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:56.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.239+0000 7fb83a9b6640 1 --2- 172.21.15.26:0/1699227083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb834102a30 0x7fb834198130 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:56.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.239+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1699227083 >> 172.21.15.26:0/1699227083 conn(0x7fb8340fc420 msgr2=0x7fb8341032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.239+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1699227083 shutdown_connections 2024-04-21T10:30:56.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.239+0000 7fb83a9b6640 1 -- 172.21.15.26:0/1699227083 wait complete. 2024-04-21T10:30:56.251 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-21T10:30:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.357+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2289749963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4100e60 msgr2=0x7f8ea4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.357+0000 7f8eaa09a640 1 --2- 172.21.15.26:0/2289749963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4100e60 0x7f8ea4101280 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009900 tx=0x7f8e9402bc30 comp rx=0 tx=0).stop 2024-04-21T10:30:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.357+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2289749963 shutdown_connections 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.357+0000 7f8eaa09a640 1 --2- 172.21.15.26:0/2289749963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ea4102a00 0x7f8ea4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.357+0000 7f8eaa09a640 1 --2- 172.21.15.26:0/2289749963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ea4102040 0x7f8ea41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.357+0000 7f8eaa09a640 1 --2- 172.21.15.26:0/2289749963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4100e60 0x7f8ea4101280 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.357+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2289749963 >> 172.21.15.26:0/2289749963 conn(0x7f8ea40fc420 msgr2=0x7f8ea40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.357+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2289749963 shutdown_connections 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.357+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2289749963 wait complete. 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8eaa09a640 1 Processor -- start 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8eaa09a640 1 -- start start 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8eaa09a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ea4100e60 0x7f8ea41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8eaa09a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4102040 0x7f8ea4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8eaa09a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ea4102a00 0x7f8ea419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8eaa09a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea410c9d0 con 0x7f8ea4102040 2024-04-21T10:30:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8eaa09a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea410c850 con 0x7f8ea4100e60 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8eaa09a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea410cb50 con 0x7f8ea4102a00 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8e9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4102040 0x7f8ea4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8e9bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4102040 0x7f8ea4198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45986/0 (socket says 172.21.15.26:45986) 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8e9bfff640 1 -- 172.21.15.26:0/2158325172 learned_addr learned my addr 172.21.15.26:0/2158325172 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ea4100e60 0x7f8ea41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.358+0000 7f8ea3fff640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ea4102a00 0x7f8ea419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.359+0000 7f8e9bfff640 1 -- 172.21.15.26:0/2158325172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ea4102a00 msgr2=0x7f8ea419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.359+0000 7f8e9bfff640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ea4102a00 0x7f8ea419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.359+0000 7f8e9bfff640 1 -- 172.21.15.26:0/2158325172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ea4100e60 msgr2=0x7f8ea41981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.359+0000 7f8e9bfff640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ea4100e60 0x7f8ea41981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.359+0000 7f8e9bfff640 1 -- 172.21.15.26:0/2158325172 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea419fda0 con 0x7f8ea4102040 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.359+0000 7f8e9bfff640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4102040 0x7f8ea4198710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00d950 tx=0x7f8e8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.359+0000 7f8ea17fa640 1 -- 172.21.15.26:0/2158325172 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c017030 con 0x7f8ea4102040 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.359+0000 7f8ea17fa640 1 -- 172.21.15.26:0/2158325172 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e8c004020 con 0x7f8ea4102040 2024-04-21T10:30:56.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.359+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2158325172 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea41a0090 con 0x7f8ea4102040 2024-04-21T10:30:56.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.359+0000 7f8ea17fa640 1 -- 172.21.15.26:0/2158325172 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c0047a0 con 0x7f8ea4102040 2024-04-21T10:30:56.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.359+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2158325172 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea41a0630 con 0x7f8ea4102040 2024-04-21T10:30:56.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.360+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2158325172 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea41a0950 con 0x7f8ea4102040 2024-04-21T10:30:56.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.361+0000 7f8ea17fa640 1 -- 172.21.15.26:0/2158325172 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f8e8c009a30 con 0x7f8ea4102040 2024-04-21T10:30:56.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.362+0000 7f8ea17fa640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8e900a3d20 0x7f8e900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:56.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.362+0000 7f8ea37fe640 1 -- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8e900a3d20 msgr2=0x7f8e900a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/72610606 2024-04-21T10:30:56.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.362+0000 7f8ea17fa640 1 -- 172.21.15.26:0/2158325172 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8248+0+0 (secure 0 0 0) 0x7f8e8c0cfe00 con 0x7f8ea4102040 2024-04-21T10:30:56.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.362+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8e900a3d20 0x7f8e900a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T10:30:56.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.366+0000 7f8ea17fa640 1 -- 172.21.15.26:0/2158325172 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e8c09c7d0 con 0x7f8ea4102040 2024-04-21T10:30:56.401 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:56 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1699227083' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-21T10:30:56.401 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:56 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:56.100+0000 7fc725fb7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T10:30:56.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:56 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1699227083' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-21T10:30:56.476 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:56 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1699227083' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-21T10:30:56.476 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:56 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:56.221+0000 7f55324f6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T10:30:56.476 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:56 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:56.372+0000 7f55324f6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T10:30:56.476 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:56 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:56.476+0000 7f55324f6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T10:30:56.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.563+0000 7f8ea37fe640 1 -- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8e900a3d20 msgr2=0x7f8e900a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/72610606 2024-04-21T10:30:56.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.563+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8e900a3d20 0x7f8e900a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T10:30:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.584+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2158325172 --> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f8ea40fe610 con 0x7f8e900a3d20 2024-04-21T10:30:56.918 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:56 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:56.601+0000 7f55324f6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T10:30:56.918 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:56 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:56.918+0000 7f55324f6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T10:30:56.947 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:56 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:56.452+0000 7f693886d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T10:30:56.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.964+0000 7f8ea37fe640 1 -- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8e900a3d20 msgr2=0x7f8e900a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/72610606 2024-04-21T10:30:56.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:56.964+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8e900a3d20 0x7f8e900a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-21T10:30:57.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:57.256+0000 7f8ea17fa640 1 -- 172.21.15.26:0/2158325172 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f8e8c0d4070 con 0x7f8ea4102040 2024-04-21T10:30:57.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:57 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1699227083' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-21T10:30:57.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:57 smithi190 ceph-mon[31423]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:30:57.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:57 smithi190 ceph-mon[31423]: Standby manager daemon z restarted 2024-04-21T10:30:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:57 smithi190 ceph-mon[31423]: Standby manager daemon z started 2024-04-21T10:30:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:57 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.190:0/3868606441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T10:30:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:57 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.190:0/3868606441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T10:30:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:57 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.190:0/3868606441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T10:30:57.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:57 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.190:0/3868606441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T10:30:57.277 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:56 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:56.402+0000 7fc725fb7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T10:30:57.277 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:56 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:56.997+0000 7fc725fb7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T10:30:57.277 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:30:57 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z[33294]: 2024-04-21T10:30:57.212+0000 7fc725fb7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T10:30:57.448 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:57 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:57.054+0000 7f693886d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T10:30:57.448 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:30:57 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y[32860]: 2024-04-21T10:30:57.273+0000 7f693886d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T10:30:57.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:57 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1699227083' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-21T10:30:57.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:57 smithi169 ceph-mon[31469]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:30:57.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:57 smithi169 ceph-mon[31469]: Standby manager daemon z restarted 2024-04-21T10:30:57.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:57 smithi169 ceph-mon[31469]: Standby manager daemon z started 2024-04-21T10:30:57.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:57 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.190:0/3868606441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T10:30:57.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:57 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.190:0/3868606441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T10:30:57.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:57 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.190:0/3868606441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T10:30:57.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:57 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.190:0/3868606441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T10:30:57.566 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:57 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1699227083' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-21T10:30:57.566 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:57 smithi026 ceph-mon[27965]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:30:57.567 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:57 smithi026 ceph-mon[27965]: Standby manager daemon z restarted 2024-04-21T10:30:57.567 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:57 smithi026 ceph-mon[27965]: Standby manager daemon z started 2024-04-21T10:30:57.567 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:57 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.190:0/3868606441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T10:30:57.567 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:57 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.190:0/3868606441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T10:30:57.567 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:57 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.190:0/3868606441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T10:30:57.567 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:57 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.190:0/3868606441' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T10:30:57.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:57.765+0000 7f8ea37fe640 1 -- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8e900a3d20 msgr2=0x7f8e900a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/72610606 2024-04-21T10:30:57.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:57.765+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8e900a3d20 0x7f8e900a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-21T10:30:57.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:57.809+0000 7f8ea17fa640 1 -- 172.21.15.26:0/2158325172 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f8e8c098740 con 0x7f8ea4102040 2024-04-21T10:30:57.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:57.810+0000 7f8ea17fa640 1 -- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8e900a3d20 msgr2=0x7f8e900a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T10:30:57.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:57.810+0000 7f8ea17fa640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/72610606,v1:172.21.15.26:6801/72610606] conn(0x7f8e900a3d20 0x7f8e900a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:57.935 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:57 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:57.566+0000 7f55324f6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T10:30:57.935 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:57 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:57.797+0000 7f55324f6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T10:30:58.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.042+0000 7f549466f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:58.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.042+0000 7f549466f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:58.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.042+0000 7f549466f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:58.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.042+0000 7f549466f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:58.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.042+0000 7f549466f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:58.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.042+0000 7f5490627700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:58.251 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.042+0000 7f5490627700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:58.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.042+0000 7f5490627700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:58.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.042+0000 7f5490627700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:58.252 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.042+0000 7f5490627700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:58.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-04-21T10:30:58.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: Standby manager daemon y restarted 2024-04-21T10:30:58.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: Standby manager daemon y started 2024-04-21T10:30:58.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.169:0/149308024' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T10:30:58.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.169:0/149308024' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T10:30:58.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.169:0/149308024' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T10:30:58.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.? 172.21.15.169:0/149308024' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T10:30:58.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: Active manager daemon x restarted 2024-04-21T10:30:58.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: Activating manager daemon x 2024-04-21T10:30:58.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: mgrmap e21: x(active, starting, since 0.0113099s), standbys: y, z 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T10:30:58.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:30:58.529 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:30:58.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T10:30:58.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T10:30:58.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T10:30:58.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: Manager daemon x is now available 2024-04-21T10:30:58.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:30:58.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T10:30:58.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:58 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T10:30:58.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-04-21T10:30:58.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: Standby manager daemon y restarted 2024-04-21T10:30:58.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: Standby manager daemon y started 2024-04-21T10:30:58.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.169:0/149308024' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T10:30:58.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.169:0/149308024' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T10:30:58.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.169:0/149308024' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T10:30:58.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.? 172.21.15.169:0/149308024' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T10:30:58.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: Active manager daemon x restarted 2024-04-21T10:30:58.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: Activating manager daemon x 2024-04-21T10:30:58.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T10:30:58.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: mgrmap e21: x(active, starting, since 0.0113099s), standbys: y, z 2024-04-21T10:30:58.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:30:58.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:30:58.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:30:58.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T10:30:58.687 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T10:30:58.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T10:30:58.689 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T10:30:58.689 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: Manager daemon x is now available 2024-04-21T10:30:58.689 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:30:58.689 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T10:30:58.689 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T10:30:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-04-21T10:30:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: Standby manager daemon y restarted 2024-04-21T10:30:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: Standby manager daemon y started 2024-04-21T10:30:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.169:0/149308024' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T10:30:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.169:0/149308024' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T10:30:58.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.169:0/149308024' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.? 172.21.15.169:0/149308024' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: Active manager daemon x restarted 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: Activating manager daemon x 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: mgrmap e21: x(active, starting, since 0.0113099s), standbys: y, z 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T10:30:58.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T10:30:58.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T10:30:58.701 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T10:30:58.701 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T10:30:58.701 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T10:30:58.701 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T10:30:58.701 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T10:30:58.701 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T10:30:58.701 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T10:30:58.701 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T10:30:58.701 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: Manager daemon x is now available 2024-04-21T10:30:58.701 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:30:58.701 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T10:30:58.701 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:58 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T10:30:58.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.818+0000 7f8ea17fa640 1 -- 172.21.15.26:0/2158325172 <== mon.0 v2:172.21.15.26:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f8e8c04d690 con 0x7f8ea4102040 2024-04-21T10:30:58.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.819+0000 7f8ea17fa640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f8e90035890 0x7f8e90035c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:58.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.819+0000 7f8ea17fa640 1 -- 172.21.15.26:0/2158325172 --> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f8e8c01ce50 con 0x7f8e90035890 2024-04-21T10:30:58.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.820+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f8e90035890 0x7f8e90035c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:58.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.823+0000 7f8ea37fe640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f8e90035890 0x7f8e90035c70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8e94005ab0 tx=0x7f8e94005fd0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:58.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.840+0000 7f8ea17fa640 1 -- 172.21.15.26:0/2158325172 <== mgr.24869 v2:172.21.15.26:6800/4245046040 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f8e8c01ce50 con 0x7f8e90035890 2024-04-21T10:30:58.839 INFO:teuthology.orchestra.run.smithi026.stdout:{} 2024-04-21T10:30:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.842+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f8e90035890 msgr2=0x7f8e90035c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.842+0000 7f8eaa09a640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f8e90035890 0x7f8e90035c70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8e94005ab0 tx=0x7f8e94005fd0 comp rx=0 tx=0).stop 2024-04-21T10:30:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.842+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4102040 msgr2=0x7f8ea4198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.842+0000 7f8eaa09a640 1 --2- 172.21.15.26:0/2158325172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea4102040 0x7f8ea4198710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00d950 tx=0x7f8e8c00de10 comp rx=0 tx=0).stop 2024-04-21T10:30:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.842+0000 7f8ea3fff640 1 -- 172.21.15.26:0/2158325172 reap_dead start 2024-04-21T10:30:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.842+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2158325172 shutdown_connections 2024-04-21T10:30:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.842+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2158325172 >> 172.21.15.26:0/2158325172 conn(0x7f8ea40fc420 msgr2=0x7f8ea40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.842+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2158325172 shutdown_connections 2024-04-21T10:30:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.842+0000 7f8eaa09a640 1 -- 172.21.15.26:0/2158325172 wait complete. 2024-04-21T10:30:58.855 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-21T10:30:58.866 INFO:teuthology.run_tasks:Running task exec... 2024-04-21T10:30:58.875 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T10:30:58.875 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-04-21T10:30:58.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-21T10:30:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 -- 172.21.15.26:0/2417608201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3768102a00 msgr2=0x7f3768109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 --2- 172.21.15.26:0/2417608201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3768102a00 0x7f3768109320 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3760009ca0 tx=0x7f376002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:30:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 -- 172.21.15.26:0/2417608201 shutdown_connections 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 --2- 172.21.15.26:0/2417608201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3768102a00 0x7f3768109320 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 --2- 172.21.15.26:0/2417608201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3768102040 0x7f37681024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 --2- 172.21.15.26:0/2417608201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3768100e60 0x7f3768101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 -- 172.21.15.26:0/2417608201 >> 172.21.15.26:0/2417608201 conn(0x7f37680fc420 msgr2=0x7f37680fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 -- 172.21.15.26:0/2417608201 shutdown_connections 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 -- 172.21.15.26:0/2417608201 wait complete. 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 Processor -- start 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 -- start start 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3768100e60 0x7f3768198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3768102040 0x7f3768198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3768102a00 0x7f376819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f376810daf0 con 0x7f3768100e60 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f376810d970 con 0x7f3768102040 2024-04-21T10:30:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.983+0000 7f376f006640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f376810dc70 con 0x7f3768102a00 2024-04-21T10:30:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f376cda2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3768100e60 0x7f3768198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f376cda2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3768100e60 0x7f3768198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46136/0 (socket says 172.21.15.26:46136) 2024-04-21T10:30:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f376cda2640 1 -- 172.21.15.26:0/695526628 learned_addr learned my addr 172.21.15.26:0/695526628 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f376d5a3640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3768102a00 0x7f376819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f3767fff640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3768102040 0x7f3768198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f376cda2640 1 -- 172.21.15.26:0/695526628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3768102a00 msgr2=0x7f376819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f376cda2640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3768102a00 0x7f376819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f376cda2640 1 -- 172.21.15.26:0/695526628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3768102040 msgr2=0x7f3768198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f376cda2640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3768102040 0x7f3768198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f376cda2640 1 -- 172.21.15.26:0/695526628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376819fd30 con 0x7f3768100e60 2024-04-21T10:30:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f376cda2640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3768100e60 0x7f3768198110 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f375c00d950 tx=0x7f375c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f3765ffb640 1 -- 172.21.15.26:0/695526628 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c017020 con 0x7f3768100e60 2024-04-21T10:30:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f3765ffb640 1 -- 172.21.15.26:0/695526628 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f375c009c30 con 0x7f3768100e60 2024-04-21T10:30:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f376f006640 1 -- 172.21.15.26:0/695526628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376819ffc0 con 0x7f3768100e60 2024-04-21T10:30:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f3765ffb640 1 -- 172.21.15.26:0/695526628 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c00f6b0 con 0x7f3768100e60 2024-04-21T10:30:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.984+0000 7f376f006640 1 -- 172.21.15.26:0/695526628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681a0480 con 0x7f3768100e60 2024-04-21T10:30:58.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.985+0000 7f376f006640 1 -- 172.21.15.26:0/695526628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3768108200 con 0x7f3768100e60 2024-04-21T10:30:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.986+0000 7f3765ffb640 1 -- 172.21.15.26:0/695526628 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f375c01a020 con 0x7f3768100e60 2024-04-21T10:30:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.987+0000 7f3765ffb640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f37580a3a60 0x7f37580a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.987+0000 7f3765ffb640 1 -- 172.21.15.26:0/695526628 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f375c0cfd80 con 0x7f3768100e60 2024-04-21T10:30:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.987+0000 7f3767fff640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f37580a3a60 0x7f37580a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.987+0000 7f3767fff640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f37580a3a60 0x7f37580a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3754005fd0 tx=0x7f3754005850 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:58.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:58.992+0000 7f3765ffb640 1 -- 172.21.15.26:0/695526628 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f375c009de0 con 0x7f3768100e60 2024-04-21T10:30:59.095 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.830+0000 7f548ee24700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:59.095 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.830+0000 7f548ee24700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:59.096 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.830+0000 7f548ee24700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:59.096 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.830+0000 7f548ee24700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:59.096 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:30:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:30:58.830+0000 7f548ee24700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:30:59.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.202+0000 7f376f006640 1 -- 172.21.15.26:0/695526628 --> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f3768062390 con 0x7f37580a3a60 2024-04-21T10:30:59.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.269+0000 7f3765ffb640 1 -- 172.21.15.26:0/695526628 <== mgr.24869 v2:172.21.15.26:6800/4245046040 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3768062390 con 0x7f37580a3a60 2024-04-21T10:30:59.268 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path / 2024-04-21T10:30:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.271+0000 7f376f006640 1 -- 172.21.15.26:0/695526628 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f37580a3a60 msgr2=0x7f37580a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.271+0000 7f376f006640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f37580a3a60 0x7f37580a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3754005fd0 tx=0x7f3754005850 comp rx=0 tx=0).stop 2024-04-21T10:30:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.271+0000 7f376f006640 1 -- 172.21.15.26:0/695526628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3768100e60 msgr2=0x7f3768198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.271+0000 7f376f006640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3768100e60 0x7f3768198110 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f375c00d950 tx=0x7f375c00de10 comp rx=0 tx=0).stop 2024-04-21T10:30:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.271+0000 7f376f006640 1 -- 172.21.15.26:0/695526628 shutdown_connections 2024-04-21T10:30:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.271+0000 7f376f006640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f37580a3a60 0x7f37580a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.271+0000 7f376f006640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3768102a00 0x7f376819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.271+0000 7f376f006640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3768102040 0x7f3768198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.271+0000 7f376f006640 1 --2- 172.21.15.26:0/695526628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3768100e60 0x7f3768198110 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.271+0000 7f376f006640 1 -- 172.21.15.26:0/695526628 >> 172.21.15.26:0/695526628 conn(0x7f37680fc420 msgr2=0x7f37681053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.271+0000 7f376f006640 1 -- 172.21.15.26:0/695526628 shutdown_connections 2024-04-21T10:30:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.271+0000 7f376f006640 1 -- 172.21.15.26:0/695526628 wait complete. 2024-04-21T10:30:59.282 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-21T10:30:59.323 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:59 smithi169 ceph-mon[31469]: [21/Apr/2024:10:30:58] ENGINE Bus STARTING 2024-04-21T10:30:59.323 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:59 smithi169 ceph-mon[31469]: [21/Apr/2024:10:30:58] ENGINE Serving on https://172.21.15.26:7150 2024-04-21T10:30:59.323 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:59 smithi169 ceph-mon[31469]: [21/Apr/2024:10:30:58] ENGINE Serving on http://172.21.15.26:8765 2024-04-21T10:30:59.323 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:59 smithi169 ceph-mon[31469]: [21/Apr/2024:10:30:58] ENGINE Bus STARTED 2024-04-21T10:30:59.323 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:30:59 smithi169 ceph-mon[31469]: mgrmap e22: x(active, since 1.02019s), standbys: y, z 2024-04-21T10:30:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.389+0000 7f570130e640 1 -- 172.21.15.26:0/1833922070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56fc107d40 msgr2=0x7f56fc10a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.389+0000 7f570130e640 1 --2- 172.21.15.26:0/1833922070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56fc107d40 0x7f56fc10a190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009ca0 tx=0x7f56f402c0f0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.389+0000 7f570130e640 1 -- 172.21.15.26:0/1833922070 shutdown_connections 2024-04-21T10:30:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.389+0000 7f570130e640 1 --2- 172.21.15.26:0/1833922070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56fc107d40 0x7f56fc10a190 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.389+0000 7f570130e640 1 --2- 172.21.15.26:0/1833922070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56fc105410 0x7f56fc107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.389+0000 7f570130e640 1 --2- 172.21.15.26:0/1833922070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56fc102ae0 0x7f56fc104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.389+0000 7f570130e640 1 -- 172.21.15.26:0/1833922070 >> 172.21.15.26:0/1833922070 conn(0x7f56fc0fc4d0 msgr2=0x7f56fc0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.389+0000 7f570130e640 1 -- 172.21.15.26:0/1833922070 shutdown_connections 2024-04-21T10:30:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.389+0000 7f570130e640 1 -- 172.21.15.26:0/1833922070 wait complete. 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f570130e640 1 Processor -- start 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f570130e640 1 -- start start 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f570130e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56fc102ae0 0x7f56fc1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f570130e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56fc105410 0x7f56fc198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f570130e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56fc107d40 0x7f56fc19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f570130e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56fc10db30 con 0x7f56fc107d40 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f570130e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56fc10d9b0 con 0x7f56fc102ae0 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f570130e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f56fc10dcb0 con 0x7f56fc105410 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f56fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56fc107d40 0x7f56fc19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f56fb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56fc107d40 0x7f56fc19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46160/0 (socket says 172.21.15.26:46160) 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f56fb7fe640 1 -- 172.21.15.26:0/57722351 learned_addr learned my addr 172.21.15.26:0/57722351 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f56fa7fc640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56fc105410 0x7f56fc198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f56faffd640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56fc102ae0 0x7f56fc1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f56fb7fe640 1 -- 172.21.15.26:0/57722351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56fc105410 msgr2=0x7f56fc198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f56fb7fe640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56fc105410 0x7f56fc198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f56fb7fe640 1 -- 172.21.15.26:0/57722351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56fc102ae0 msgr2=0x7f56fc1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f56fb7fe640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56fc102ae0 0x7f56fc1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.390+0000 7f56fb7fe640 1 -- 172.21.15.26:0/57722351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56fc19fce0 con 0x7f56fc107d40 2024-04-21T10:30:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.391+0000 7f56fb7fe640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56fc107d40 0x7f56fc19f670 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f56f402c700 tx=0x7f56f4034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.391+0000 7f56e3fff640 1 -- 172.21.15.26:0/57722351 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f4039040 con 0x7f56fc107d40 2024-04-21T10:30:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.391+0000 7f56e3fff640 1 -- 172.21.15.26:0/57722351 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56f4044070 con 0x7f56fc107d40 2024-04-21T10:30:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.391+0000 7f570130e640 1 -- 172.21.15.26:0/57722351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56fc19ff70 con 0x7f56fc107d40 2024-04-21T10:30:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.391+0000 7f56e3fff640 1 -- 172.21.15.26:0/57722351 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f40042a0 con 0x7f56fc107d40 2024-04-21T10:30:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.391+0000 7f570130e640 1 -- 172.21.15.26:0/57722351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56fc1a03d0 con 0x7f56fc107d40 2024-04-21T10:30:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.392+0000 7f570130e640 1 -- 172.21.15.26:0/57722351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56fc0fdfc0 con 0x7f56fc107d40 2024-04-21T10:30:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.393+0000 7f56e3fff640 1 -- 172.21.15.26:0/57722351 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f56f4003970 con 0x7f56fc107d40 2024-04-21T10:30:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.393+0000 7f56e3fff640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f56e40a3a60 0x7f56e40a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.394+0000 7f56faffd640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f56e40a3a60 0x7f56e40a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.394+0000 7f56e3fff640 1 -- 172.21.15.26:0/57722351 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f56f40f24d0 con 0x7f56fc107d40 2024-04-21T10:30:59.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.394+0000 7f56faffd640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f56e40a3a60 0x7f56e40a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f56ec005fd0 tx=0x7f56ec005850 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.398+0000 7f56e3fff640 1 -- 172.21.15.26:0/57722351 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56f40bf090 con 0x7f56fc107d40 2024-04-21T10:30:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:59 smithi026 ceph-mon[27965]: [21/Apr/2024:10:30:58] ENGINE Bus STARTING 2024-04-21T10:30:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:59 smithi026 ceph-mon[27965]: [21/Apr/2024:10:30:58] ENGINE Serving on https://172.21.15.26:7150 2024-04-21T10:30:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:59 smithi026 ceph-mon[27965]: [21/Apr/2024:10:30:58] ENGINE Serving on http://172.21.15.26:8765 2024-04-21T10:30:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:59 smithi026 ceph-mon[27965]: [21/Apr/2024:10:30:58] ENGINE Bus STARTED 2024-04-21T10:30:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:30:59 smithi026 ceph-mon[27965]: mgrmap e22: x(active, since 1.02019s), standbys: y, z 2024-04-21T10:30:59.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:59 smithi190 ceph-mon[31423]: [21/Apr/2024:10:30:58] ENGINE Bus STARTING 2024-04-21T10:30:59.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:59 smithi190 ceph-mon[31423]: [21/Apr/2024:10:30:58] ENGINE Serving on https://172.21.15.26:7150 2024-04-21T10:30:59.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:59 smithi190 ceph-mon[31423]: [21/Apr/2024:10:30:58] ENGINE Serving on http://172.21.15.26:8765 2024-04-21T10:30:59.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:59 smithi190 ceph-mon[31423]: [21/Apr/2024:10:30:58] ENGINE Bus STARTED 2024-04-21T10:30:59.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:30:59 smithi190 ceph-mon[31423]: mgrmap e22: x(active, since 1.02019s), standbys: y, z 2024-04-21T10:30:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.617+0000 7f570130e640 1 -- 172.21.15.26:0/57722351 --> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f56fc0624f0 con 0x7f56e40a3a60 2024-04-21T10:30:59.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.682+0000 7f56e3fff640 1 -- 172.21.15.26:0/57722351 <== mgr.24869 v2:172.21.15.26:6800/4245046040 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f56fc0624f0 con 0x7f56e40a3a60 2024-04-21T10:30:59.682 INFO:teuthology.orchestra.run.smithi026.stdout:Retention added to path / 2024-04-21T10:30:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.684+0000 7f570130e640 1 -- 172.21.15.26:0/57722351 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f56e40a3a60 msgr2=0x7f56e40a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.684+0000 7f570130e640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f56e40a3a60 0x7f56e40a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f56ec005fd0 tx=0x7f56ec005850 comp rx=0 tx=0).stop 2024-04-21T10:30:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.684+0000 7f570130e640 1 -- 172.21.15.26:0/57722351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56fc107d40 msgr2=0x7f56fc19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.684+0000 7f570130e640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56fc107d40 0x7f56fc19f670 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f56f402c700 tx=0x7f56f4034760 comp rx=0 tx=0).stop 2024-04-21T10:30:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.684+0000 7f570130e640 1 -- 172.21.15.26:0/57722351 shutdown_connections 2024-04-21T10:30:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.684+0000 7f570130e640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f56e40a3a60 0x7f56e40a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.684+0000 7f570130e640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56fc107d40 0x7f56fc19f670 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.684+0000 7f570130e640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56fc105410 0x7f56fc198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.684+0000 7f570130e640 1 --2- 172.21.15.26:0/57722351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56fc102ae0 0x7f56fc1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.684+0000 7f570130e640 1 -- 172.21.15.26:0/57722351 >> 172.21.15.26:0/57722351 conn(0x7f56fc0fc4d0 msgr2=0x7f56fc103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.684+0000 7f570130e640 1 -- 172.21.15.26:0/57722351 shutdown_connections 2024-04-21T10:30:59.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.684+0000 7f570130e640 1 -- 172.21.15.26:0/57722351 wait complete. 2024-04-21T10:30:59.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-21T10:30:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.808+0000 7f5052596640 1 -- 172.21.15.26:0/916688327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f504c107cd0 msgr2=0x7f504c10a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.808+0000 7f5052596640 1 --2- 172.21.15.26:0/916688327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f504c107cd0 0x7f504c10a120 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5044009ca0 tx=0x7f504402c0f0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.808+0000 7f5052596640 1 -- 172.21.15.26:0/916688327 shutdown_connections 2024-04-21T10:30:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.808+0000 7f5052596640 1 --2- 172.21.15.26:0/916688327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f504c107cd0 0x7f504c10a120 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.808+0000 7f5052596640 1 --2- 172.21.15.26:0/916688327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c1053a0 0x7f504c107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.808+0000 7f5052596640 1 --2- 172.21.15.26:0/916688327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f504c102a70 0x7f504c104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.808+0000 7f5052596640 1 -- 172.21.15.26:0/916688327 >> 172.21.15.26:0/916688327 conn(0x7f504c0fc460 msgr2=0x7f504c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:30:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.808+0000 7f5052596640 1 -- 172.21.15.26:0/916688327 shutdown_connections 2024-04-21T10:30:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.808+0000 7f5052596640 1 -- 172.21.15.26:0/916688327 wait complete. 2024-04-21T10:30:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f5052596640 1 Processor -- start 2024-04-21T10:30:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f5052596640 1 -- start start 2024-04-21T10:30:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f5052596640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c102a70 0x7f504c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f5052596640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f504c1053a0 0x7f504c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f5052596640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f504c107cd0 0x7f504c1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f5052596640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10db30 con 0x7f504c107cd0 2024-04-21T10:30:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f5052596640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10d9b0 con 0x7f504c1053a0 2024-04-21T10:30:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f5052596640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10dcb0 con 0x7f504c102a70 2024-04-21T10:30:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f5050b33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f504c107cd0 0x7f504c1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f504b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f504c1053a0 0x7f504c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f504bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c102a70 0x7f504c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f504b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f504c1053a0 0x7f504c19a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.26:48768/0 (socket says 172.21.15.26:48768) 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f5050b33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f504c107cd0 0x7f504c1a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46180/0 (socket says 172.21.15.26:46180) 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.809+0000 7f504b7fe640 1 -- 172.21.15.26:0/3024612641 learned_addr learned my addr 172.21.15.26:0/3024612641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.810+0000 7f5050b33640 1 -- 172.21.15.26:0/3024612641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c102a70 msgr2=0x7f504c19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.810+0000 7f5050b33640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c102a70 0x7f504c19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.810+0000 7f5050b33640 1 -- 172.21.15.26:0/3024612641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f504c1053a0 msgr2=0x7f504c19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.810+0000 7f5050b33640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f504c1053a0 0x7f504c19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.810+0000 7f5050b33640 1 -- 172.21.15.26:0/3024612641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c1a1fe0 con 0x7f504c107cd0 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.810+0000 7f504b7fe640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f504c1053a0 0x7f504c19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.810+0000 7f504bfff640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c102a70 0x7f504c19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.810+0000 7f5050b33640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f504c107cd0 0x7f504c1a1970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5044009bf0 tx=0x7f5044034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:59.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.811+0000 7f50497fa640 1 -- 172.21.15.26:0/3024612641 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044041020 con 0x7f504c107cd0 2024-04-21T10:30:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.811+0000 7f50497fa640 1 -- 172.21.15.26:0/3024612641 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5044034ca0 con 0x7f504c107cd0 2024-04-21T10:30:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.811+0000 7f50497fa640 1 -- 172.21.15.26:0/3024612641 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044039040 con 0x7f504c107cd0 2024-04-21T10:30:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.811+0000 7f5052596640 1 -- 172.21.15.26:0/3024612641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504c1a2270 con 0x7f504c107cd0 2024-04-21T10:30:59.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.811+0000 7f5052596640 1 -- 172.21.15.26:0/3024612641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504c1a26e0 con 0x7f504c107cd0 2024-04-21T10:30:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.812+0000 7f5052596640 1 -- 172.21.15.26:0/3024612641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504c1097c0 con 0x7f504c107cd0 2024-04-21T10:30:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.813+0000 7f50497fa640 1 -- 172.21.15.26:0/3024612641 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f5044044020 con 0x7f504c107cd0 2024-04-21T10:30:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.814+0000 7f50497fa640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f50340a3a60 0x7f50340a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:30:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.814+0000 7f504bfff640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f50340a3a60 0x7f50340a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:30:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.814+0000 7f50497fa640 1 -- 172.21.15.26:0/3024612641 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f50440f1d40 con 0x7f504c107cd0 2024-04-21T10:30:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.815+0000 7f504bfff640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f50340a3a60 0x7f50340a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f503c0095f0 tx=0x7f503c005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:30:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:30:59.819+0000 7f50497fa640 1 -- 172.21.15.26:0/3024612641 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f504404b020 con 0x7f504c107cd0 2024-04-21T10:31:00.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.089+0000 7f5052596640 1 -- 172.21.15.26:0/3024612641 --> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f504c062390 con 0x7f50340a3a60 2024-04-21T10:31:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.097+0000 7f50497fa640 1 -- 172.21.15.26:0/3024612641 <== mgr.24869 v2:172.21.15.26:6800/4245046040 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f504c062390 con 0x7f50340a3a60 2024-04-21T10:31:00.097 INFO:teuthology.orchestra.run.smithi026.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-21T00:00:00", "created": "2024-04-21T10:30:59", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-21T10:31:00.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.099+0000 7f5052596640 1 -- 172.21.15.26:0/3024612641 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f50340a3a60 msgr2=0x7f50340a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:00.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.099+0000 7f5052596640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f50340a3a60 0x7f50340a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f503c0095f0 tx=0x7f503c005f90 comp rx=0 tx=0).stop 2024-04-21T10:31:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.099+0000 7f5052596640 1 -- 172.21.15.26:0/3024612641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f504c107cd0 msgr2=0x7f504c1a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.099+0000 7f5052596640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f504c107cd0 0x7f504c1a1970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5044009bf0 tx=0x7f5044034670 comp rx=0 tx=0).stop 2024-04-21T10:31:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.100+0000 7f5052596640 1 -- 172.21.15.26:0/3024612641 shutdown_connections 2024-04-21T10:31:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.100+0000 7f5052596640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f50340a3a60 0x7f50340a5f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.100+0000 7f5052596640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f504c107cd0 0x7f504c1a1970 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.100+0000 7f5052596640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f504c1053a0 0x7f504c19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.100+0000 7f5052596640 1 --2- 172.21.15.26:0/3024612641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f504c102a70 0x7f504c19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.100+0000 7f5052596640 1 -- 172.21.15.26:0/3024612641 >> 172.21.15.26:0/3024612641 conn(0x7f504c0fc460 msgr2=0x7f504c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.100+0000 7f5052596640 1 -- 172.21.15.26:0/3024612641 shutdown_connections 2024-04-21T10:31:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.100+0000 7f5052596640 1 -- 172.21.15.26:0/3024612641 wait complete. 2024-04-21T10:31:00.110 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-21T10:31:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 -- 172.21.15.26:0/2007524284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54001053e0 msgr2=0x7f54001077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 --2- 172.21.15.26:0/2007524284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54001053e0 0x7f54001077d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009900 tx=0x7f53fc02bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 -- 172.21.15.26:0/2007524284 shutdown_connections 2024-04-21T10:31:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 --2- 172.21.15.26:0/2007524284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5400107d10 0x7f540010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 --2- 172.21.15.26:0/2007524284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54001053e0 0x7f54001077d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 --2- 172.21.15.26:0/2007524284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5400102ab0 0x7f5400104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 -- 172.21.15.26:0/2007524284 >> 172.21.15.26:0/2007524284 conn(0x7f54000fc420 msgr2=0x7f54000fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 -- 172.21.15.26:0/2007524284 shutdown_connections 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 -- 172.21.15.26:0/2007524284 wait complete. 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 Processor -- start 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 -- start start 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5400102ab0 0x7f54001981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54001053e0 0x7f5400198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5400107d10 0x7f540019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f540010daf0 con 0x7f54001053e0 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f540010d970 con 0x7f5400102ab0 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.216+0000 7f540806a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f540010dc70 con 0x7f5400107d10 2024-04-21T10:31:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f5405605640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54001053e0 0x7f5400198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f5405605640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54001053e0 0x7f5400198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46202/0 (socket says 172.21.15.26:46202) 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f5405605640 1 -- 172.21.15.26:0/3491218912 learned_addr learned my addr 172.21.15.26:0/3491218912 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f5405e06640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5400102ab0 0x7f54001981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f5406607640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5400107d10 0x7f540019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f5405605640 1 -- 172.21.15.26:0/3491218912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5400107d10 msgr2=0x7f540019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f5405605640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5400107d10 0x7f540019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f5405605640 1 -- 172.21.15.26:0/3491218912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5400102ab0 msgr2=0x7f54001981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f5405605640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5400102ab0 0x7f54001981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f5405605640 1 -- 172.21.15.26:0/3491218912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540019fdf0 con 0x7f54001053e0 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f5405605640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54001053e0 0x7f5400198710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f53fc000c00 tx=0x7f53fc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f53f6ffd640 1 -- 172.21.15.26:0/3491218912 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc043020 con 0x7f54001053e0 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f53f6ffd640 1 -- 172.21.15.26:0/3491218912 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53fc004350 con 0x7f54001053e0 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f540806a640 1 -- 172.21.15.26:0/3491218912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54001a0080 con 0x7f54001053e0 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f53f6ffd640 1 -- 172.21.15.26:0/3491218912 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc003ac0 con 0x7f54001053e0 2024-04-21T10:31:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.217+0000 7f540806a640 1 -- 172.21.15.26:0/3491218912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54001a0620 con 0x7f54001053e0 2024-04-21T10:31:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.218+0000 7f540806a640 1 -- 172.21.15.26:0/3491218912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54000fdf10 con 0x7f54001053e0 2024-04-21T10:31:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.220+0000 7f53f6ffd640 1 -- 172.21.15.26:0/3491218912 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f53fc046020 con 0x7f54001053e0 2024-04-21T10:31:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.220+0000 7f53f6ffd640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f53ec0a3ae0 0x7f53ec0a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.220+0000 7f5405e06640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f53ec0a3ae0 0x7f53ec0a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.220+0000 7f53f6ffd640 1 -- 172.21.15.26:0/3491218912 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f53fc0f20a0 con 0x7f54001053e0 2024-04-21T10:31:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.221+0000 7f5405e06640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f53ec0a3ae0 0x7f53ec0a5fa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009bd0 tx=0x7f53f0005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:00.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.229+0000 7f53f6ffd640 1 -- 172.21.15.26:0/3491218912 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53fc04d020 con 0x7f54001053e0 2024-04-21T10:31:00.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.269+0000 7f53f6ffd640 1 -- 172.21.15.26:0/3491218912 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f53fc0baca0 con 0x7f54001053e0 2024-04-21T10:31:00.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-mon[27965]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:00.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-mon[27965]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:31:00.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-mon[27965]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:00.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.505+0000 7f540806a640 1 -- 172.21.15.26:0/3491218912 --> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5400062390 con 0x7f53ec0a3ae0 2024-04-21T10:31:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.524+0000 7f53f6ffd640 1 -- 172.21.15.26:0/3491218912 <== mgr.24869 v2:172.21.15.26:6800/4245046040 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5400062390 con 0x7f53ec0a3ae0 2024-04-21T10:31:00.523 INFO:teuthology.orchestra.run.smithi026.stdout:/ 1m 6m3h 2024-04-21T10:31:00.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.525+0000 7f540806a640 1 -- 172.21.15.26:0/3491218912 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f53ec0a3ae0 msgr2=0x7f53ec0a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:00.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.525+0000 7f540806a640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f53ec0a3ae0 0x7f53ec0a5fa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009bd0 tx=0x7f53f0005f90 comp rx=0 tx=0).stop 2024-04-21T10:31:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.525+0000 7f540806a640 1 -- 172.21.15.26:0/3491218912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54001053e0 msgr2=0x7f5400198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.525+0000 7f540806a640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54001053e0 0x7f5400198710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f53fc000c00 tx=0x7f53fc005dc0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.526+0000 7f540806a640 1 -- 172.21.15.26:0/3491218912 shutdown_connections 2024-04-21T10:31:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.526+0000 7f540806a640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f53ec0a3ae0 0x7f53ec0a5fa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.526+0000 7f540806a640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5400107d10 0x7f540019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.526+0000 7f540806a640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54001053e0 0x7f5400198710 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.526+0000 7f540806a640 1 --2- 172.21.15.26:0/3491218912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5400102ab0 0x7f54001981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.526+0000 7f540806a640 1 -- 172.21.15.26:0/3491218912 >> 172.21.15.26:0/3491218912 conn(0x7f54000fc420 msgr2=0x7f5400103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.526+0000 7f540806a640 1 -- 172.21.15.26:0/3491218912 shutdown_connections 2024-04-21T10:31:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.526+0000 7f540806a640 1 -- 172.21.15.26:0/3491218912 wait complete. 2024-04-21T10:31:00.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:00 smithi190 ceph-mon[31423]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:00.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:00 smithi190 ceph-mon[31423]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:31:00.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:00 smithi190 ceph-mon[31423]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:00.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:00 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:00 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:00 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:00 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:00 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:00 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.537 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-21T10:31:00.566 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-21T10:31:00.577 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-21T10:31:00.612 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-04-21T10:31:00.612 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-21T10:31:00.614 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-21T10:31:00.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:31:00.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:00 smithi169 ceph-mon[31469]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:00.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:00 smithi169 ceph-mon[31469]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:31:00.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:00 smithi169 ceph-mon[31469]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:00.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:00 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:00 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:00 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:00 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:00 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:00 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:00.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.744+0000 7f61a18b3640 1 -- 172.21.15.26:0/444923532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f619c105d60 msgr2=0x7f619c10c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.744+0000 7f61a18b3640 1 --2- 172.21.15.26:0/444923532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f619c105d60 0x7f619c10c5f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6194009ca0 tx=0x7f619402c0f0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.744+0000 7f61a18b3640 1 -- 172.21.15.26:0/444923532 shutdown_connections 2024-04-21T10:31:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.744+0000 7f61a18b3640 1 --2- 172.21.15.26:0/444923532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f619c105d60 0x7f619c10c5f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.744+0000 7f61a18b3640 1 --2- 172.21.15.26:0/444923532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c1053a0 0x7f619c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.744+0000 7f61a18b3640 1 --2- 172.21.15.26:0/444923532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f619c1041b0 0x7f619c1045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.744+0000 7f61a18b3640 1 -- 172.21.15.26:0/444923532 >> 172.21.15.26:0/444923532 conn(0x7f619c0ff740 msgr2=0x7f619c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.744+0000 7f61a18b3640 1 -- 172.21.15.26:0/444923532 shutdown_connections 2024-04-21T10:31:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.744+0000 7f61a18b3640 1 -- 172.21.15.26:0/444923532 wait complete. 2024-04-21T10:31:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.745+0000 7f61a18b3640 1 Processor -- start 2024-04-21T10:31:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.745+0000 7f61a18b3640 1 -- start start 2024-04-21T10:31:00.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.745+0000 7f61a18b3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c1041b0 0x7f619c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:00.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.745+0000 7f61a18b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f619c1053a0 0x7f619c19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.745+0000 7f61a18b3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f619c105d60 0x7f619c1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.745+0000 7f61a18b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f619c10fcc0 con 0x7f619c1053a0 2024-04-21T10:31:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.745+0000 7f61a18b3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f619c10fb40 con 0x7f619c1041b0 2024-04-21T10:31:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.745+0000 7f61a18b3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f619c10fe40 con 0x7f619c105d60 2024-04-21T10:31:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f619affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c1041b0 0x7f619c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f619a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f619c1053a0 0x7f619c19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f619affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c1041b0 0x7f619c19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.26:48858/0 (socket says 172.21.15.26:48858) 2024-04-21T10:31:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f619a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f619c1053a0 0x7f619c19a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46254/0 (socket says 172.21.15.26:46254) 2024-04-21T10:31:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f619affd640 1 -- 172.21.15.26:0/3182992134 learned_addr learned my addr 172.21.15.26:0/3182992134 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f619b7fe640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f619c105d60 0x7f619c1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f619affd640 1 -- 172.21.15.26:0/3182992134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f619c105d60 msgr2=0x7f619c1a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f619affd640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f619c105d60 0x7f619c1a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f619affd640 1 -- 172.21.15.26:0/3182992134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f619c1053a0 msgr2=0x7f619c19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f619affd640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f619c1053a0 0x7f619c19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f619affd640 1 -- 172.21.15.26:0/3182992134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c1a1f20 con 0x7f619c1041b0 2024-04-21T10:31:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f619affd640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c1041b0 0x7f619c19a300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f618c00e9c0 tx=0x7f618c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.746+0000 7f61a08b1640 1 -- 172.21.15.26:0/3182992134 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c00cd80 con 0x7f619c1041b0 2024-04-21T10:31:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.747+0000 7f61a18b3640 1 -- 172.21.15.26:0/3182992134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619c1a2210 con 0x7f619c1041b0 2024-04-21T10:31:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.747+0000 7f61a08b1640 1 -- 172.21.15.26:0/3182992134 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f618c0035f0 con 0x7f619c1041b0 2024-04-21T10:31:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.747+0000 7f61a08b1640 1 -- 172.21.15.26:0/3182992134 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c010690 con 0x7f619c1041b0 2024-04-21T10:31:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.747+0000 7f61a18b3640 1 -- 172.21.15.26:0/3182992134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619c1a2760 con 0x7f619c1041b0 2024-04-21T10:31:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.748+0000 7f61a18b3640 1 -- 172.21.15.26:0/3182992134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f619c10b550 con 0x7f619c1041b0 2024-04-21T10:31:00.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.749+0000 7f61a08b1640 1 -- 172.21.15.26:0/3182992134 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f618c0037a0 con 0x7f619c1041b0 2024-04-21T10:31:00.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.749+0000 7f61a08b1640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f61840a3db0 0x7f61840a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:00.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.750+0000 7f619a7fc640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f61840a3db0 0x7f61840a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:00.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.750+0000 7f61a08b1640 1 -- 172.21.15.26:0/3182992134 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f618c0d0e00 con 0x7f619c1041b0 2024-04-21T10:31:00.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.750+0000 7f619a7fc640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f61840a3db0 0x7f61840a6270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f619c19b810 tx=0x7f6190005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:00.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.758+0000 7f61a08b1640 1 -- 172.21.15.26:0/3182992134 <== mon.2 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f618c09d340 con 0x7f619c1041b0 2024-04-21T10:31:00.935 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:31:00.716+0000 7f546e1a3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:31:00.935 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:31:00.716+0000 7f546e1a3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:31:00.935 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:31:00.716+0000 7f546e1a3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:31:00.935 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:31:00.716+0000 7f546e1a3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:31:00.935 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:31:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:31:00.716+0000 7f546e1a3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T10:31:00.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.958+0000 7f61a18b3640 1 -- 172.21.15.26:0/3182992134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f619c062b80 con 0x7f619c1041b0 2024-04-21T10:31:00.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.961+0000 7f61a08b1640 1 -- 172.21.15.26:0/3182992134 <== mon.2 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+23079 (secure 0 0 0) 0x7f618c09cc40 con 0x7f619c1041b0 2024-04-21T10:31:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:00.961 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":90,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:30:57.801035+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3802401516":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/3125048023":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4245538524":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2797499963":"2024-04-22T10:30:57.800969+0000","172.21.15.26:6801/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:31:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.964+0000 7f61a18b3640 1 -- 172.21.15.26:0/3182992134 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f61840a3db0 msgr2=0x7f61840a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.965+0000 7f61a18b3640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f61840a3db0 0x7f61840a6270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f619c19b810 tx=0x7f6190005f90 comp rx=0 tx=0).stop 2024-04-21T10:31:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.965+0000 7f61a18b3640 1 -- 172.21.15.26:0/3182992134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c1041b0 msgr2=0x7f619c19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.965+0000 7f61a18b3640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c1041b0 0x7f619c19a300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f618c00e9c0 tx=0x7f618c00ee80 comp rx=0 tx=0).stop 2024-04-21T10:31:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.965+0000 7f61a18b3640 1 -- 172.21.15.26:0/3182992134 shutdown_connections 2024-04-21T10:31:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.965+0000 7f61a18b3640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f61840a3db0 0x7f61840a6270 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.965+0000 7f61a18b3640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f619c105d60 0x7f619c1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.965+0000 7f61a18b3640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f619c1053a0 0x7f619c19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.965+0000 7f61a18b3640 1 --2- 172.21.15.26:0/3182992134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f619c1041b0 0x7f619c19a300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.965+0000 7f61a18b3640 1 -- 172.21.15.26:0/3182992134 >> 172.21.15.26:0/3182992134 conn(0x7f619c0ff740 msgr2=0x7f619c108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.965+0000 7f61a18b3640 1 -- 172.21.15.26:0/3182992134 shutdown_connections 2024-04-21T10:31:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:00.965+0000 7f61a18b3640 1 -- 172.21.15.26:0/3182992134 wait complete. 2024-04-21T10:31:00.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T10:24:28.176891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T10:28:59.417687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T10:29:00.234808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T10:31:00.977 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T10:31:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.084+0000 7f04abd0e640 1 -- 172.21.15.26:0/3347078528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a4101fe0 msgr2=0x7f04a4102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.084+0000 7f04abd0e640 1 --2- 172.21.15.26:0/3347078528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a4101fe0 0x7f04a4102460 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009900 tx=0x7f04a002bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.085+0000 7f04abd0e640 1 -- 172.21.15.26:0/3347078528 shutdown_connections 2024-04-21T10:31:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.085+0000 7f04abd0e640 1 --2- 172.21.15.26:0/3347078528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a41029a0 0x7f04a41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.085+0000 7f04abd0e640 1 --2- 172.21.15.26:0/3347078528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a4101fe0 0x7f04a4102460 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.085+0000 7f04abd0e640 1 --2- 172.21.15.26:0/3347078528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a4100e90 0x7f04a41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.085+0000 7f04abd0e640 1 -- 172.21.15.26:0/3347078528 >> 172.21.15.26:0/3347078528 conn(0x7f04a40fc400 msgr2=0x7f04a40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.085+0000 7f04abd0e640 1 -- 172.21.15.26:0/3347078528 shutdown_connections 2024-04-21T10:31:01.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.085+0000 7f04abd0e640 1 -- 172.21.15.26:0/3347078528 wait complete. 2024-04-21T10:31:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.085+0000 7f04abd0e640 1 Processor -- start 2024-04-21T10:31:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.085+0000 7f04abd0e640 1 -- start start 2024-04-21T10:31:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04abd0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a4100e90 0x7f04a4198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04abd0e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a4101fe0 0x7f04a4198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04abd0e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a41029a0 0x7f04a419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04abd0e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04a410c9f0 con 0x7f04a4100e90 2024-04-21T10:31:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04abd0e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04a410c870 con 0x7f04a4101fe0 2024-04-21T10:31:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04abd0e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f04a410cb70 con 0x7f04a41029a0 2024-04-21T10:31:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04a9aaa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a4100e90 0x7f04a4198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04a9aaa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a4100e90 0x7f04a4198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46270/0 (socket says 172.21.15.26:46270) 2024-04-21T10:31:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04a9aaa640 1 -- 172.21.15.26:0/536568124 learned_addr learned my addr 172.21.15.26:0/536568124 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04a92a9640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a4101fe0 0x7f04a4198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04aa2ab640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a41029a0 0x7f04a419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04a9aaa640 1 -- 172.21.15.26:0/536568124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a41029a0 msgr2=0x7f04a419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04a9aaa640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a41029a0 0x7f04a419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04a9aaa640 1 -- 172.21.15.26:0/536568124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a4101fe0 msgr2=0x7f04a4198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04a9aaa640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a4101fe0 0x7f04a4198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04a9aaa640 1 -- 172.21.15.26:0/536568124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a419fe70 con 0x7f04a4100e90 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.086+0000 7f04a9aaa640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a4100e90 0x7f04a4198250 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f049400e9f0 tx=0x7f049400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.087+0000 7f049affd640 1 -- 172.21.15.26:0/536568124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049400cd80 con 0x7f04a4100e90 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.087+0000 7f049affd640 1 -- 172.21.15.26:0/536568124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04940054d0 con 0x7f04a4100e90 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.087+0000 7f049affd640 1 -- 172.21.15.26:0/536568124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494010690 con 0x7f04a4100e90 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.087+0000 7f04abd0e640 1 -- 172.21.15.26:0/536568124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a41a0160 con 0x7f04a4100e90 2024-04-21T10:31:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.087+0000 7f04abd0e640 1 -- 172.21.15.26:0/536568124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a41a0630 con 0x7f04a4100e90 2024-04-21T10:31:01.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.088+0000 7f04abd0e640 1 -- 172.21.15.26:0/536568124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a41a08f0 con 0x7f04a4100e90 2024-04-21T10:31:01.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.089+0000 7f049affd640 1 -- 172.21.15.26:0/536568124 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f0494004e90 con 0x7f04a4100e90 2024-04-21T10:31:01.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.090+0000 7f049affd640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f04900a3db0 0x7f04900a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:01.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.090+0000 7f04a92a9640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f04900a3db0 0x7f04900a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:01.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.090+0000 7f049affd640 1 -- 172.21.15.26:0/536568124 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f04940d0df0 con 0x7f04a4100e90 2024-04-21T10:31:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.090+0000 7f04a92a9640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f04900a3db0 0x7f04900a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f04a0005ab0 tx=0x7f04a0005fd0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.094+0000 7f049affd640 1 -- 172.21.15.26:0/536568124 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f049409d2d0 con 0x7f04a4100e90 2024-04-21T10:31:01.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.290+0000 7f04abd0e640 1 -- 172.21.15.26:0/536568124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04a41a0ac0 con 0x7f04a4100e90 2024-04-21T10:31:01.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.290+0000 7f049affd640 1 -- 172.21.15.26:0/536568124 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f049409cbd0 con 0x7f04a4100e90 2024-04-21T10:31:01.290 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-21T10:31:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.292+0000 7f04abd0e640 1 -- 172.21.15.26:0/536568124 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f04900a3db0 msgr2=0x7f04900a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.292+0000 7f04abd0e640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f04900a3db0 0x7f04900a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f04a0005ab0 tx=0x7f04a0005fd0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.292+0000 7f04abd0e640 1 -- 172.21.15.26:0/536568124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a4100e90 msgr2=0x7f04a4198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.292+0000 7f04abd0e640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a4100e90 0x7f04a4198250 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f049400e9f0 tx=0x7f049400edf0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.292+0000 7f04abd0e640 1 -- 172.21.15.26:0/536568124 shutdown_connections 2024-04-21T10:31:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.292+0000 7f04abd0e640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f04900a3db0 0x7f04900a6270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.292+0000 7f04abd0e640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04a41029a0 0x7f04a419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.292+0000 7f04abd0e640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04a4101fe0 0x7f04a4198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.293+0000 7f04abd0e640 1 --2- 172.21.15.26:0/536568124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04a4100e90 0x7f04a4198250 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.293+0000 7f04abd0e640 1 -- 172.21.15.26:0/536568124 >> 172.21.15.26:0/536568124 conn(0x7f04a40fc400 msgr2=0x7f04a4069130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.293+0000 7f04abd0e640 1 -- 172.21.15.26:0/536568124 shutdown_connections 2024-04-21T10:31:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.293+0000 7f04abd0e640 1 -- 172.21.15.26:0/536568124 wait complete. 2024-04-21T10:31:01.303 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T10:31:01.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.425+0000 7ff3b4221640 1 -- 172.21.15.26:0/1098145908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac100e90 msgr2=0x7ff3ac1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.425+0000 7ff3b4221640 1 --2- 172.21.15.26:0/1098145908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac100e90 0x7ff3ac1012b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8009900 tx=0x7ff3a802bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.426+0000 7ff3b4221640 1 -- 172.21.15.26:0/1098145908 shutdown_connections 2024-04-21T10:31:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.426+0000 7ff3b4221640 1 --2- 172.21.15.26:0/1098145908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3ac1029a0 0x7ff3ac1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.426+0000 7ff3b4221640 1 --2- 172.21.15.26:0/1098145908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ac101fe0 0x7ff3ac102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.426+0000 7ff3b4221640 1 --2- 172.21.15.26:0/1098145908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac100e90 0x7ff3ac1012b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.426+0000 7ff3b4221640 1 -- 172.21.15.26:0/1098145908 >> 172.21.15.26:0/1098145908 conn(0x7ff3ac0fc400 msgr2=0x7ff3ac0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.426+0000 7ff3b4221640 1 -- 172.21.15.26:0/1098145908 shutdown_connections 2024-04-21T10:31:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.426+0000 7ff3b4221640 1 -- 172.21.15.26:0/1098145908 wait complete. 2024-04-21T10:31:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b4221640 1 Processor -- start 2024-04-21T10:31:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b4221640 1 -- start start 2024-04-21T10:31:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b4221640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3ac100e90 0x7ff3ac1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b4221640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ac101fe0 0x7ff3ac198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b4221640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac1029a0 0x7ff3ac19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b4221640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac10ca30 con 0x7ff3ac1029a0 2024-04-21T10:31:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b4221640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac10c8b0 con 0x7ff3ac100e90 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b4221640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac10cbb0 con 0x7ff3ac101fe0 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b27be640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac1029a0 0x7ff3ac19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b27be640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac1029a0 0x7ff3ac19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46290/0 (socket says 172.21.15.26:46290) 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b27be640 1 -- 172.21.15.26:0/2882481533 learned_addr learned my addr 172.21.15.26:0/2882481533 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b1fbd640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3ac100e90 0x7ff3ac1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b17bc640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ac101fe0 0x7ff3ac198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b27be640 1 -- 172.21.15.26:0/2882481533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ac101fe0 msgr2=0x7ff3ac198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b27be640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ac101fe0 0x7ff3ac198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b27be640 1 -- 172.21.15.26:0/2882481533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3ac100e90 msgr2=0x7ff3ac1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b27be640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3ac100e90 0x7ff3ac1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.427+0000 7ff3b27be640 1 -- 172.21.15.26:0/2882481533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac19fe90 con 0x7ff3ac1029a0 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.428+0000 7ff3b27be640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac1029a0 0x7ff3ac19f820 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400eb10 tx=0x7ff3a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.428+0000 7ff3a2ffd640 1 -- 172.21.15.26:0/2882481533 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4004b20 con 0x7ff3ac1029a0 2024-04-21T10:31:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.428+0000 7ff3a2ffd640 1 -- 172.21.15.26:0/2882481533 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a4004cc0 con 0x7ff3ac1029a0 2024-04-21T10:31:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.428+0000 7ff3b4221640 1 -- 172.21.15.26:0/2882481533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ac1a0180 con 0x7ff3ac1029a0 2024-04-21T10:31:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.428+0000 7ff3a2ffd640 1 -- 172.21.15.26:0/2882481533 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4010670 con 0x7ff3ac1029a0 2024-04-21T10:31:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.428+0000 7ff3b4221640 1 -- 172.21.15.26:0/2882481533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ac1a0680 con 0x7ff3ac1029a0 2024-04-21T10:31:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.429+0000 7ff3b4221640 1 -- 172.21.15.26:0/2882481533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3ac1a09a0 con 0x7ff3ac1029a0 2024-04-21T10:31:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.430+0000 7ff3a2ffd640 1 -- 172.21.15.26:0/2882481533 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7ff3a4005020 con 0x7ff3ac1029a0 2024-04-21T10:31:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.431+0000 7ff3a2ffd640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff3980a3db0 0x7ff3980a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.431+0000 7ff3a2ffd640 1 -- 172.21.15.26:0/2882481533 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7ff3a40d0f50 con 0x7ff3ac1029a0 2024-04-21T10:31:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.431+0000 7ff3b1fbd640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff3980a3db0 0x7ff3980a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.432+0000 7ff3b1fbd640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff3980a3db0 0x7ff3980a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8000c00 tx=0x7ff3a802c7e0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.436+0000 7ff3a2ffd640 1 -- 172.21.15.26:0/2882481533 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3a409d490 con 0x7ff3ac1029a0 2024-04-21T10:31:01.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3182992134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.528 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:01 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.628+0000 7ff3b4221640 1 -- 172.21.15.26:0/2882481533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff3ac1a0b70 con 0x7ff3ac1029a0 2024-04-21T10:31:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.628+0000 7ff3a2ffd640 1 -- 172.21.15.26:0/2882481533 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7ff3a409cd90 con 0x7ff3ac1029a0 2024-04-21T10:31:01.628 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-21T10:31:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.631+0000 7ff3b4221640 1 -- 172.21.15.26:0/2882481533 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff3980a3db0 msgr2=0x7ff3980a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.631+0000 7ff3b4221640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff3980a3db0 0x7ff3980a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8000c00 tx=0x7ff3a802c7e0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.631+0000 7ff3b4221640 1 -- 172.21.15.26:0/2882481533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac1029a0 msgr2=0x7ff3ac19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.631+0000 7ff3b4221640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac1029a0 0x7ff3ac19f820 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400eb10 tx=0x7ff3a400c2d0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.631+0000 7ff3b4221640 1 -- 172.21.15.26:0/2882481533 shutdown_connections 2024-04-21T10:31:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.631+0000 7ff3b4221640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff3980a3db0 0x7ff3980a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.631+0000 7ff3b4221640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3ac1029a0 0x7ff3ac19f820 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.631+0000 7ff3b4221640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3ac101fe0 0x7ff3ac198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.631+0000 7ff3b4221640 1 --2- 172.21.15.26:0/2882481533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3ac100e90 0x7ff3ac1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.631+0000 7ff3b4221640 1 -- 172.21.15.26:0/2882481533 >> 172.21.15.26:0/2882481533 conn(0x7ff3ac0fc400 msgr2=0x7ff3ac0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.631+0000 7ff3b4221640 1 -- 172.21.15.26:0/2882481533 shutdown_connections 2024-04-21T10:31:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.631+0000 7ff3b4221640 1 -- 172.21.15.26:0/2882481533 wait complete. 2024-04-21T10:31:01.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:01.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:31:01.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:01.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-21T10:31:01.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:01.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3182992134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:31:01.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:01.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:01.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.642 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.643 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:01.643 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:01.643 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.643 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.643 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.643 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:01 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.644 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T10:31:01.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:01.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:31:01.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3182992134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:01 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.751+0000 7f4935c43640 1 -- 172.21.15.26:0/2226124758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930100800 msgr2=0x7f4930100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.751+0000 7f4935c43640 1 --2- 172.21.15.26:0/2226124758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930100800 0x7f4930100c20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4920009900 tx=0x7f492002bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.751+0000 7f4935c43640 1 -- 172.21.15.26:0/2226124758 shutdown_connections 2024-04-21T10:31:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.751+0000 7f4935c43640 1 --2- 172.21.15.26:0/2226124758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4930101b00 0x7f4930101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.751+0000 7f4935c43640 1 --2- 172.21.15.26:0/2226124758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4930101160 0x7f49301015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.751+0000 7f4935c43640 1 --2- 172.21.15.26:0/2226124758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930100800 0x7f4930100c20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.751+0000 7f4935c43640 1 -- 172.21.15.26:0/2226124758 >> 172.21.15.26:0/2226124758 conn(0x7f49300fc400 msgr2=0x7f49300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.751+0000 7f4935c43640 1 -- 172.21.15.26:0/2226124758 shutdown_connections 2024-04-21T10:31:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.751+0000 7f4935c43640 1 -- 172.21.15.26:0/2226124758 wait complete. 2024-04-21T10:31:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f4935c43640 1 Processor -- start 2024-04-21T10:31:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f4935c43640 1 -- start start 2024-04-21T10:31:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f4935c43640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4930100800 0x7f493019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f4935c43640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930101160 0x7f493019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f4935c43640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4930101b00 0x7f49301a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f4935c43640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f493010fd30 con 0x7f4930101160 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f4935c43640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f493010fbb0 con 0x7f4930100800 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f4935c43640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f493010feb0 con 0x7f4930101b00 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f492effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930101160 0x7f493019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f492effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930101160 0x7f493019a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46298/0 (socket says 172.21.15.26:46298) 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f492effd640 1 -- 172.21.15.26:0/3657748404 learned_addr learned my addr 172.21.15.26:0/3657748404 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f492f7fe640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4930100800 0x7f493019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f492ffff640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4930101b00 0x7f49301a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f492effd640 1 -- 172.21.15.26:0/3657748404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4930101b00 msgr2=0x7f49301a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f492effd640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4930101b00 0x7f49301a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f492effd640 1 -- 172.21.15.26:0/3657748404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4930100800 msgr2=0x7f493019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f492effd640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4930100800 0x7f493019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.752+0000 7f492effd640 1 -- 172.21.15.26:0/3657748404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49301a2020 con 0x7f4930101160 2024-04-21T10:31:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.753+0000 7f492effd640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930101160 0x7f493019a940 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f492400e9f0 tx=0x7f492400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.753+0000 7f492cff9640 1 -- 172.21.15.26:0/3657748404 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492400cd80 con 0x7f4930101160 2024-04-21T10:31:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.753+0000 7f4935c43640 1 -- 172.21.15.26:0/3657748404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49301a2310 con 0x7f4930101160 2024-04-21T10:31:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.753+0000 7f492cff9640 1 -- 172.21.15.26:0/3657748404 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49240108e0 con 0x7f4930101160 2024-04-21T10:31:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.753+0000 7f492cff9640 1 -- 172.21.15.26:0/3657748404 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4924018980 con 0x7f4930101160 2024-04-21T10:31:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.753+0000 7f4935c43640 1 -- 172.21.15.26:0/3657748404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49301a28b0 con 0x7f4930101160 2024-04-21T10:31:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.754+0000 7f4935c43640 1 -- 172.21.15.26:0/3657748404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f493010b7b0 con 0x7f4930101160 2024-04-21T10:31:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.755+0000 7f492cff9640 1 -- 172.21.15.26:0/3657748404 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f4924010430 con 0x7f4930101160 2024-04-21T10:31:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.756+0000 7f492cff9640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f49180a3db0 0x7f49180a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.756+0000 7f492cff9640 1 -- 172.21.15.26:0/3657748404 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f49240d1120 con 0x7f4930101160 2024-04-21T10:31:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.756+0000 7f492f7fe640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f49180a3db0 0x7f49180a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.757+0000 7f492f7fe640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f49180a3db0 0x7f49180a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4920000c00 tx=0x7f492002c7e0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:01.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.760+0000 7f492cff9640 1 -- 172.21.15.26:0/3657748404 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f492409d010 con 0x7f4930101160 2024-04-21T10:31:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.960+0000 7f4935c43640 1 -- 172.21.15.26:0/3657748404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f493002d5c0 con 0x7f4930101160 2024-04-21T10:31:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.960+0000 7f492cff9640 1 -- 172.21.15.26:0/3657748404 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f49240a1310 con 0x7f4930101160 2024-04-21T10:31:01.960 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-21T10:31:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.962+0000 7f4935c43640 1 -- 172.21.15.26:0/3657748404 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f49180a3db0 msgr2=0x7f49180a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.962+0000 7f4935c43640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f49180a3db0 0x7f49180a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4920000c00 tx=0x7f492002c7e0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.962+0000 7f4935c43640 1 -- 172.21.15.26:0/3657748404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930101160 msgr2=0x7f493019a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.962+0000 7f4935c43640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930101160 0x7f493019a940 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f492400e9f0 tx=0x7f492400eeb0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.962+0000 7f4935c43640 1 -- 172.21.15.26:0/3657748404 shutdown_connections 2024-04-21T10:31:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.962+0000 7f4935c43640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f49180a3db0 0x7f49180a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.962+0000 7f4935c43640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4930101b00 0x7f49301a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.962+0000 7f4935c43640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4930101160 0x7f493019a940 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.963+0000 7f4935c43640 1 --2- 172.21.15.26:0/3657748404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4930100800 0x7f493019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.963+0000 7f4935c43640 1 -- 172.21.15.26:0/3657748404 >> 172.21.15.26:0/3657748404 conn(0x7f49300fc400 msgr2=0x7f49300fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.963+0000 7f4935c43640 1 -- 172.21.15.26:0/3657748404 shutdown_connections 2024-04-21T10:31:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:01.963+0000 7f4935c43640 1 -- 172.21.15.26:0/3657748404 wait complete. 2024-04-21T10:31:01.974 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:31:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.081+0000 7f1028598640 1 -- 172.21.15.26:0/3239599449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1020101b60 msgr2=0x7f102010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.081+0000 7f1028598640 1 --2- 172.21.15.26:0/3239599449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1020101b60 0x7f102010dc90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1018009cd0 tx=0x7f101802c0f0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.081+0000 7f1028598640 1 -- 172.21.15.26:0/3239599449 shutdown_connections 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.081+0000 7f1028598640 1 --2- 172.21.15.26:0/3239599449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1020101b60 0x7f102010dc90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.081+0000 7f1028598640 1 --2- 172.21.15.26:0/3239599449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201011a0 0x7f1020101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.081+0000 7f1028598640 1 --2- 172.21.15.26:0/3239599449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1020105e30 0x7f1020106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.081+0000 7f1028598640 1 -- 172.21.15.26:0/3239599449 >> 172.21.15.26:0/3239599449 conn(0x7f1020077e30 msgr2=0x7f1020078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.081+0000 7f1028598640 1 -- 172.21.15.26:0/3239599449 shutdown_connections 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.081+0000 7f1028598640 1 -- 172.21.15.26:0/3239599449 wait complete. 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.081+0000 7f1028598640 1 Processor -- start 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1028598640 1 -- start start 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1028598640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201011a0 0x7f10201986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1028598640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1020101b60 0x7f1020198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1028598640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1020105e30 0x7f102019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1028598640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10201110c0 con 0x7f1020101b60 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1028598640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1020110f40 con 0x7f10201011a0 2024-04-21T10:31:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1028598640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1020111240 con 0x7f1020105e30 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1025b33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1020101b60 0x7f1020198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1025b33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1020101b60 0x7f1020198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46312/0 (socket says 172.21.15.26:46312) 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1025b33640 1 -- 172.21.15.26:0/4103551741 learned_addr learned my addr 172.21.15.26:0/4103551741 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1026334640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201011a0 0x7f10201986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1026b35640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1020105e30 0x7f102019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1025b33640 1 -- 172.21.15.26:0/4103551741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1020105e30 msgr2=0x7f102019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1025b33640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1020105e30 0x7f102019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1025b33640 1 -- 172.21.15.26:0/4103551741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201011a0 msgr2=0x7f10201986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1025b33640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201011a0 0x7f10201986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1025b33640 1 -- 172.21.15.26:0/4103551741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102019d6a0 con 0x7f1020101b60 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1026334640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201011a0 0x7f10201986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f1025b33640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1020101b60 0x7f1020198bf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f101000e9f0 tx=0x7f101000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.082+0000 7f10177fe640 1 -- 172.21.15.26:0/4103551741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101000cd80 con 0x7f1020101b60 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.083+0000 7f10177fe640 1 -- 172.21.15.26:0/4103551741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10100108e0 con 0x7f1020101b60 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.083+0000 7f1028598640 1 -- 172.21.15.26:0/4103551741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102019d990 con 0x7f1020101b60 2024-04-21T10:31:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.083+0000 7f10177fe640 1 -- 172.21.15.26:0/4103551741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010018980 con 0x7f1020101b60 2024-04-21T10:31:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.083+0000 7f1028598640 1 -- 172.21.15.26:0/4103551741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10201a5220 con 0x7f1020101b60 2024-04-21T10:31:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.084+0000 7f1028598640 1 -- 172.21.15.26:0/4103551741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1020068de0 con 0x7f1020101b60 2024-04-21T10:31:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.085+0000 7f10177fe640 1 -- 172.21.15.26:0/4103551741 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f1010010430 con 0x7f1020101b60 2024-04-21T10:31:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.085+0000 7f10177fe640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f100c0a3db0 0x7f100c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.085+0000 7f1026334640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f100c0a3db0 0x7f100c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.086+0000 7f10177fe640 1 -- 172.21.15.26:0/4103551741 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f10100d0c50 con 0x7f1020101b60 2024-04-21T10:31:02.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.086+0000 7f1026334640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f100c0a3db0 0x7f100c0a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f101c00bfd0 tx=0x7f101c00b390 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:02.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.090+0000 7f10177fe640 1 -- 172.21.15.26:0/4103551741 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10100d6020 con 0x7f1020101b60 2024-04-21T10:31:02.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.287+0000 7f1028598640 1 -- 172.21.15.26:0/4103551741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f102010a3c0 con 0x7f1020101b60 2024-04-21T10:31:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.290+0000 7f10177fe640 1 -- 172.21.15.26:0/4103551741 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+23079 (secure 0 0 0) 0x7f101009cb40 con 0x7f1020101b60 2024-04-21T10:31:02.290 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:02.290 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":90,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:30:57.801035+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3802401516":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/3125048023":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4245538524":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2797499963":"2024-04-22T10:30:57.800969+0000","172.21.15.26:6801/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:31:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.293+0000 7f1028598640 1 -- 172.21.15.26:0/4103551741 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f100c0a3db0 msgr2=0x7f100c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.293+0000 7f1028598640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f100c0a3db0 0x7f100c0a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f101c00bfd0 tx=0x7f101c00b390 comp rx=0 tx=0).stop 2024-04-21T10:31:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.293+0000 7f1028598640 1 -- 172.21.15.26:0/4103551741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1020101b60 msgr2=0x7f1020198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.293+0000 7f1028598640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1020101b60 0x7f1020198bf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f101000e9f0 tx=0x7f101000eeb0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.294+0000 7f1028598640 1 -- 172.21.15.26:0/4103551741 shutdown_connections 2024-04-21T10:31:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.294+0000 7f1028598640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f100c0a3db0 0x7f100c0a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.294+0000 7f1028598640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1020105e30 0x7f102019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.294+0000 7f1028598640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1020101b60 0x7f1020198bf0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.294+0000 7f1028598640 1 --2- 172.21.15.26:0/4103551741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10201011a0 0x7f10201986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.294+0000 7f1028598640 1 -- 172.21.15.26:0/4103551741 >> 172.21.15.26:0/4103551741 conn(0x7f1020077e30 msgr2=0x7f10200693e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.294+0000 7f1028598640 1 -- 172.21.15.26:0/4103551741 shutdown_connections 2024-04-21T10:31:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.294+0000 7f1028598640 1 -- 172.21.15.26:0/4103551741 wait complete. 2024-04-21T10:31:02.313 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T10:24:28.176891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T10:28:59.417687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T10:29:00.234808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T10:31:02.313 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T10:31:02.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.424+0000 7f16837ef640 1 -- 172.21.15.26:0/2229830051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c101090 msgr2=0x7f167c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.424+0000 7f16837ef640 1 --2- 172.21.15.26:0/2229830051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c101090 0x7f167c101470 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1678009900 tx=0x7f167802bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.424+0000 7f16837ef640 1 -- 172.21.15.26:0/2229830051 shutdown_connections 2024-04-21T10:31:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.424+0000 7f16837ef640 1 --2- 172.21.15.26:0/2229830051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f167c106160 0x7f167c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.424+0000 7f16837ef640 1 --2- 172.21.15.26:0/2229830051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f167c101a40 0x7f167c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.424+0000 7f16837ef640 1 --2- 172.21.15.26:0/2229830051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c101090 0x7f167c101470 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.424+0000 7f16837ef640 1 -- 172.21.15.26:0/2229830051 >> 172.21.15.26:0/2229830051 conn(0x7f167c077f50 msgr2=0x7f167c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f16837ef640 1 -- 172.21.15.26:0/2229830051 shutdown_connections 2024-04-21T10:31:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f16837ef640 1 -- 172.21.15.26:0/2229830051 wait complete. 2024-04-21T10:31:02.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f16837ef640 1 Processor -- start 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f16837ef640 1 -- start start 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f16837ef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f167c101090 0x7f167c19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f16837ef640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f167c106160 0x7f167c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f16837ef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c1985f0 0x7f167c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f16837ef640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1110a0 con 0x7f167c1985f0 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f16837ef640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f167c110f20 con 0x7f167c106160 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f16837ef640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f167c111220 con 0x7f167c101090 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f1681d8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c1985f0 0x7f167c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f1681d8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c1985f0 0x7f167c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46336/0 (socket says 172.21.15.26:46336) 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.425+0000 7f1681d8c640 1 -- 172.21.15.26:0/3323333739 learned_addr learned my addr 172.21.15.26:0/3323333739 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f168158b640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f167c101090 0x7f167c19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f1680d8a640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f167c106160 0x7f167c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:02.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f1681d8c640 1 -- 172.21.15.26:0/3323333739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f167c101090 msgr2=0x7f167c19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f1681d8c640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f167c101090 0x7f167c19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f1681d8c640 1 -- 172.21.15.26:0/3323333739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f167c106160 msgr2=0x7f167c19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f1681d8c640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f167c106160 0x7f167c19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f1681d8c640 1 -- 172.21.15.26:0/3323333739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c199110 con 0x7f167c1985f0 2024-04-21T10:31:02.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f1681d8c640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c1985f0 0x7f167c198aa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f167400eb10 tx=0x7f167400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:02.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f16727fc640 1 -- 172.21.15.26:0/3323333739 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1674004b20 con 0x7f167c1985f0 2024-04-21T10:31:02.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f16837ef640 1 -- 172.21.15.26:0/3323333739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167c199400 con 0x7f167c1985f0 2024-04-21T10:31:02.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f16837ef640 1 -- 172.21.15.26:0/3323333739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167c1a5650 con 0x7f167c1985f0 2024-04-21T10:31:02.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f16727fc640 1 -- 172.21.15.26:0/3323333739 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1674004cc0 con 0x7f167c1985f0 2024-04-21T10:31:02.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.426+0000 7f16727fc640 1 -- 172.21.15.26:0/3323333739 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1674010690 con 0x7f167c1985f0 2024-04-21T10:31:02.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.427+0000 7f16837ef640 1 -- 172.21.15.26:0/3323333739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1660005290 con 0x7f167c1985f0 2024-04-21T10:31:02.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.428+0000 7f16727fc640 1 -- 172.21.15.26:0/3323333739 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f1674005000 con 0x7f167c1985f0 2024-04-21T10:31:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.429+0000 7f16727fc640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f16680a3ed0 0x7f16680a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.429+0000 7f168158b640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f16680a3ed0 0x7f16680a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.429+0000 7f16727fc640 1 -- 172.21.15.26:0/3323333739 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f16740d0ee0 con 0x7f167c1985f0 2024-04-21T10:31:02.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.430+0000 7f168158b640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f16680a3ed0 0x7f16680a6390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1678006010 tx=0x7f1678005e40 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:02.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.434+0000 7f16727fc640 1 -- 172.21.15.26:0/3323333739 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f167409d350 con 0x7f167c1985f0 2024-04-21T10:31:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.625+0000 7f16837ef640 1 -- 172.21.15.26:0/3323333739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1660005850 con 0x7f167c1985f0 2024-04-21T10:31:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.626+0000 7f16727fc640 1 -- 172.21.15.26:0/3323333739 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f167409cc50 con 0x7f167c1985f0 2024-04-21T10:31:02.626 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-21T10:31:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.628+0000 7f16837ef640 1 -- 172.21.15.26:0/3323333739 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f16680a3ed0 msgr2=0x7f16680a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.628+0000 7f16837ef640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f16680a3ed0 0x7f16680a6390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1678006010 tx=0x7f1678005e40 comp rx=0 tx=0).stop 2024-04-21T10:31:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.628+0000 7f16837ef640 1 -- 172.21.15.26:0/3323333739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c1985f0 msgr2=0x7f167c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.628+0000 7f16837ef640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c1985f0 0x7f167c198aa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f167400eb10 tx=0x7f167400c2d0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.628+0000 7f16837ef640 1 -- 172.21.15.26:0/3323333739 shutdown_connections 2024-04-21T10:31:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.628+0000 7f16837ef640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f16680a3ed0 0x7f16680a6390 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.628+0000 7f16837ef640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f167c1985f0 0x7f167c198aa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.628+0000 7f16837ef640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f167c106160 0x7f167c19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.628+0000 7f16837ef640 1 --2- 172.21.15.26:0/3323333739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f167c101090 0x7f167c19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.628+0000 7f16837ef640 1 -- 172.21.15.26:0/3323333739 >> 172.21.15.26:0/3323333739 conn(0x7f167c077f50 msgr2=0x7f167c0695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.628+0000 7f16837ef640 1 -- 172.21.15.26:0/3323333739 shutdown_connections 2024-04-21T10:31:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.628+0000 7f16837ef640 1 -- 172.21.15.26:0/3323333739 wait complete. 2024-04-21T10:31:02.637 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi190 to 176.2M 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: Unable to set osd_memory_target on smithi190 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: Unable to set osd_memory_target on smithi169 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: Unable to set osd_memory_target on smithi026 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:31:02.638 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/536568124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:31:02.639 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:31:02.639 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: Updating smithi026:/etc/ceph/ceph.conf 2024-04-21T10:31:02.639 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-21T10:31:02.639 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: Updating smithi190:/etc/ceph/ceph.conf 2024-04-21T10:31:02.639 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2882481533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:31:02.639 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:02 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3657748404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:31:02.640 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T10:31:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi190 to 176.2M 2024-04-21T10:31:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: Unable to set osd_memory_target on smithi190 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-21T10:31:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-21T10:31:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: Unable to set osd_memory_target on smithi169 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-21T10:31:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: Unable to set osd_memory_target on smithi026 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/536568124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: Updating smithi026:/etc/ceph/ceph.conf 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: Updating smithi190:/etc/ceph/ceph.conf 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2882481533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:31:02.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:02 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3657748404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:31:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.749+0000 7f040aed2640 1 -- 172.21.15.26:0/3869668949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04041010a0 msgr2=0x7f0404101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.749+0000 7f040aed2640 1 --2- 172.21.15.26:0/3869668949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04041010a0 0x7f0404101480 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009900 tx=0x7f03f402bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.749+0000 7f040aed2640 1 -- 172.21.15.26:0/3869668949 shutdown_connections 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.749+0000 7f040aed2640 1 --2- 172.21.15.26:0/3869668949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0404106170 0x7f040410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.749+0000 7f040aed2640 1 --2- 172.21.15.26:0/3869668949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0404101a50 0x7f0404105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.749+0000 7f040aed2640 1 --2- 172.21.15.26:0/3869668949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04041010a0 0x7f0404101480 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.749+0000 7f040aed2640 1 -- 172.21.15.26:0/3869668949 >> 172.21.15.26:0/3869668949 conn(0x7f0404077eb0 msgr2=0x7f04040782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.749+0000 7f040aed2640 1 -- 172.21.15.26:0/3869668949 shutdown_connections 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.749+0000 7f040aed2640 1 -- 172.21.15.26:0/3869668949 wait complete. 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.749+0000 7f040aed2640 1 Processor -- start 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.749+0000 7f040aed2640 1 -- start start 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f040aed2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04041010a0 0x7f04041986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f040aed2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0404106170 0x7f0404198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f040aed2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f040419cf90 0x7f040419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f040aed2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04041121f0 con 0x7f0404106170 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f040aed2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0404112070 con 0x7f04041010a0 2024-04-21T10:31:02.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f040aed2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0404112370 con 0x7f040419cf90 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0403fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0404106170 0x7f0404198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0403fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0404106170 0x7f0404198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46362/0 (socket says 172.21.15.26:46362) 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0403fff640 1 -- 172.21.15.26:0/3094969291 learned_addr learned my addr 172.21.15.26:0/3094969291 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0408c6e640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04041010a0 0x7f04041986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f040946f640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f040419cf90 0x7f040419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0403fff640 1 -- 172.21.15.26:0/3094969291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f040419cf90 msgr2=0x7f040419d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0403fff640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f040419cf90 0x7f040419d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0403fff640 1 -- 172.21.15.26:0/3094969291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04041010a0 msgr2=0x7f04041986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0403fff640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04041010a0 0x7f04041986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0403fff640 1 -- 172.21.15.26:0/3094969291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040419da80 con 0x7f0404106170 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f040946f640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f040419cf90 0x7f040419d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0408c6e640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04041010a0 0x7f04041986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:02.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0403fff640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0404106170 0x7f0404198c10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f03f800c970 tx=0x7f03f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:02.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0401ffb640 1 -- 172.21.15.26:0/3094969291 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8007bb0 con 0x7f0404106170 2024-04-21T10:31:02.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.750+0000 7f0401ffb640 1 -- 172.21.15.26:0/3094969291 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f8007d50 con 0x7f0404106170 2024-04-21T10:31:02.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.751+0000 7f040aed2640 1 -- 172.21.15.26:0/3094969291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040419dd70 con 0x7f0404106170 2024-04-21T10:31:02.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.751+0000 7f0401ffb640 1 -- 172.21.15.26:0/3094969291 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f800f450 con 0x7f0404106170 2024-04-21T10:31:02.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.751+0000 7f040aed2640 1 -- 172.21.15.26:0/3094969291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04041a5560 con 0x7f0404106170 2024-04-21T10:31:02.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.752+0000 7f040aed2640 1 -- 172.21.15.26:0/3094969291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04040691c0 con 0x7f0404106170 2024-04-21T10:31:02.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.753+0000 7f0401ffb640 1 -- 172.21.15.26:0/3094969291 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f03f8004f20 con 0x7f0404106170 2024-04-21T10:31:02.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.753+0000 7f0401ffb640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f03ec0a3ed0 0x7f03ec0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:02.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.753+0000 7f0408c6e640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f03ec0a3ed0 0x7f03ec0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:02.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.753+0000 7f0401ffb640 1 -- 172.21.15.26:0/3094969291 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f03f80d0b00 con 0x7f0404106170 2024-04-21T10:31:02.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.754+0000 7f0408c6e640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f03ec0a3ed0 0x7f03ec0a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f03f4006010 tx=0x7f03f4005e40 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:02.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.758+0000 7f0401ffb640 1 -- 172.21.15.26:0/3094969291 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03f8014030 con 0x7f0404106170 2024-04-21T10:31:02.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi190 to 176.2M 2024-04-21T10:31:02.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: Unable to set osd_memory_target on smithi190 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: Unable to set osd_memory_target on smithi169 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: Unable to set osd_memory_target on smithi026 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/536568124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:31:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:31:02.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: Updating smithi026:/etc/ceph/ceph.conf 2024-04-21T10:31:02.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-21T10:31:02.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: Updating smithi190:/etc/ceph/ceph.conf 2024-04-21T10:31:02.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2882481533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:31:02.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:02 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3657748404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:31:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.945+0000 7f040aed2640 1 -- 172.21.15.26:0/3094969291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0404101a50 con 0x7f0404106170 2024-04-21T10:31:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.946+0000 7f0401ffb640 1 -- 172.21.15.26:0/3094969291 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f03f809cf70 con 0x7f0404106170 2024-04-21T10:31:02.945 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-21T10:31:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.948+0000 7f040aed2640 1 -- 172.21.15.26:0/3094969291 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f03ec0a3ed0 msgr2=0x7f03ec0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.948+0000 7f040aed2640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f03ec0a3ed0 0x7f03ec0a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f03f4006010 tx=0x7f03f4005e40 comp rx=0 tx=0).stop 2024-04-21T10:31:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.948+0000 7f040aed2640 1 -- 172.21.15.26:0/3094969291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0404106170 msgr2=0x7f0404198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.948+0000 7f040aed2640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0404106170 0x7f0404198c10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f03f800c970 tx=0x7f03f800ce30 comp rx=0 tx=0).stop 2024-04-21T10:31:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.948+0000 7f040aed2640 1 -- 172.21.15.26:0/3094969291 shutdown_connections 2024-04-21T10:31:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.948+0000 7f040aed2640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f03ec0a3ed0 0x7f03ec0a6390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.948+0000 7f040aed2640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f040419cf90 0x7f040419d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.948+0000 7f040aed2640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0404106170 0x7f0404198c10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.948+0000 7f040aed2640 1 --2- 172.21.15.26:0/3094969291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04041010a0 0x7f04041986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.948+0000 7f040aed2640 1 -- 172.21.15.26:0/3094969291 >> 172.21.15.26:0/3094969291 conn(0x7f0404077eb0 msgr2=0x7f04040ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.948+0000 7f040aed2640 1 -- 172.21.15.26:0/3094969291 shutdown_connections 2024-04-21T10:31:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:02.948+0000 7f040aed2640 1 -- 172.21.15.26:0/3094969291 wait complete. 2024-04-21T10:31:02.958 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T10:31:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.063+0000 7f1c39ca5640 1 -- 172.21.15.26:0/2309593753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c3410c510 msgr2=0x7f1c3410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.063+0000 7f1c39ca5640 1 --2- 172.21.15.26:0/2309593753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c3410c510 0x7f1c3410e9b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009ca0 tx=0x7f1c2c02c230 comp rx=0 tx=0).stop 2024-04-21T10:31:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.063+0000 7f1c39ca5640 1 -- 172.21.15.26:0/2309593753 shutdown_connections 2024-04-21T10:31:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.063+0000 7f1c39ca5640 1 --2- 172.21.15.26:0/2309593753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c3410c510 0x7f1c3410e9b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.063+0000 7f1c39ca5640 1 --2- 172.21.15.26:0/2309593753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c34103af0 0x7f1c3410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.063+0000 7f1c39ca5640 1 --2- 172.21.15.26:0/2309593753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c341031d0 0x7f1c341035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.063+0000 7f1c39ca5640 1 -- 172.21.15.26:0/2309593753 >> 172.21.15.26:0/2309593753 conn(0x7f1c34077f50 msgr2=0x7f1c34078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:03.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.064+0000 7f1c39ca5640 1 -- 172.21.15.26:0/2309593753 shutdown_connections 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.064+0000 7f1c39ca5640 1 -- 172.21.15.26:0/2309593753 wait complete. 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.064+0000 7f1c39ca5640 1 Processor -- start 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.064+0000 7f1c39ca5640 1 -- start start 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.064+0000 7f1c39ca5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c341031d0 0x7f1c34100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.064+0000 7f1c39ca5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c34103af0 0x7f1c34101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.064+0000 7f1c39ca5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c3410c510 0x7f1c341018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.064+0000 7f1c39ca5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34112180 con 0x7f1c341031d0 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.064+0000 7f1c39ca5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34112000 con 0x7f1c3410c510 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.064+0000 7f1c39ca5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34112300 con 0x7f1c34103af0 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c341031d0 0x7f1c34100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c341031d0 0x7f1c34100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46390/0 (socket says 172.21.15.26:46390) 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c337fe640 1 -- 172.21.15.26:0/3453980540 learned_addr learned my addr 172.21.15.26:0/3453980540 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c32ffd640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c34103af0 0x7f1c34101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c337fe640 1 -- 172.21.15.26:0/3453980540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c34103af0 msgr2=0x7f1c34101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c33fff640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c3410c510 0x7f1c341018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c337fe640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c34103af0 0x7f1c34101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c337fe640 1 -- 172.21.15.26:0/3453980540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c3410c510 msgr2=0x7f1c341018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c337fe640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c3410c510 0x7f1c341018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c337fe640 1 -- 172.21.15.26:0/3453980540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c341a6be0 con 0x7f1c341031d0 2024-04-21T10:31:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c337fe640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c341031d0 0x7f1c34100e20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800d950 tx=0x7f1c2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c30ff9640 1 -- 172.21.15.26:0/3453980540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28017070 con 0x7f1c341031d0 2024-04-21T10:31:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c30ff9640 1 -- 172.21.15.26:0/3453980540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c28009c30 con 0x7f1c341031d0 2024-04-21T10:31:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c39ca5640 1 -- 172.21.15.26:0/3453980540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c341a6ed0 con 0x7f1c341031d0 2024-04-21T10:31:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c30ff9640 1 -- 172.21.15.26:0/3453980540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800f620 con 0x7f1c341031d0 2024-04-21T10:31:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.065+0000 7f1c39ca5640 1 -- 172.21.15.26:0/3453980540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c341a7390 con 0x7f1c341031d0 2024-04-21T10:31:03.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.066+0000 7f1c39ca5640 1 -- 172.21.15.26:0/3453980540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c34068d70 con 0x7f1c341031d0 2024-04-21T10:31:03.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.067+0000 7f1c30ff9640 1 -- 172.21.15.26:0/3453980540 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f1c28010460 con 0x7f1c341031d0 2024-04-21T10:31:03.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.068+0000 7f1c30ff9640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1c1c0a3ed0 0x7f1c1c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:03.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.068+0000 7f1c32ffd640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1c1c0a3ed0 0x7f1c1c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:03.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.069+0000 7f1c30ff9640 1 -- 172.21.15.26:0/3453980540 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f1c280d07a0 con 0x7f1c341031d0 2024-04-21T10:31:03.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.069+0000 7f1c32ffd640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1c1c0a3ed0 0x7f1c1c0a6390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005d90 tx=0x7f1c24005d20 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:03.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.073+0000 7f1c30ff9640 1 -- 172.21.15.26:0/3453980540 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c2809cc10 con 0x7f1c341031d0 2024-04-21T10:31:03.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.260+0000 7f1c39ca5640 1 -- 172.21.15.26:0/3453980540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c34107100 con 0x7f1c341031d0 2024-04-21T10:31:03.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.260+0000 7f1c30ff9640 1 -- 172.21.15.26:0/3453980540 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f1c28009de0 con 0x7f1c341031d0 2024-04-21T10:31:03.260 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-21T10:31:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.262+0000 7f1c39ca5640 1 -- 172.21.15.26:0/3453980540 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1c1c0a3ed0 msgr2=0x7f1c1c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.262+0000 7f1c39ca5640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1c1c0a3ed0 0x7f1c1c0a6390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005d90 tx=0x7f1c24005d20 comp rx=0 tx=0).stop 2024-04-21T10:31:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.262+0000 7f1c39ca5640 1 -- 172.21.15.26:0/3453980540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c341031d0 msgr2=0x7f1c34100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.262+0000 7f1c39ca5640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c341031d0 0x7f1c34100e20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800d950 tx=0x7f1c2800de10 comp rx=0 tx=0).stop 2024-04-21T10:31:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.262+0000 7f1c39ca5640 1 -- 172.21.15.26:0/3453980540 shutdown_connections 2024-04-21T10:31:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.262+0000 7f1c39ca5640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1c1c0a3ed0 0x7f1c1c0a6390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.262+0000 7f1c39ca5640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c3410c510 0x7f1c341018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.262+0000 7f1c39ca5640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c34103af0 0x7f1c34101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.262+0000 7f1c39ca5640 1 --2- 172.21.15.26:0/3453980540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c341031d0 0x7f1c34100e20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.262+0000 7f1c39ca5640 1 -- 172.21.15.26:0/3453980540 >> 172.21.15.26:0/3453980540 conn(0x7f1c34077f50 msgr2=0x7f1c340fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:03.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.262+0000 7f1c39ca5640 1 -- 172.21.15.26:0/3453980540 shutdown_connections 2024-04-21T10:31:03.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.262+0000 7f1c39ca5640 1 -- 172.21.15.26:0/3453980540 wait complete. 2024-04-21T10:31:03.273 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-21T10:31:03.273 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T10:31:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.376+0000 7ff1864fd640 1 -- 172.21.15.26:0/137748768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18010c5a0 msgr2=0x7ff18010e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.376+0000 7ff1864fd640 1 --2- 172.21.15.26:0/137748768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18010c5a0 0x7ff18010e9d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff178009ca0 tx=0x7ff17802c180 comp rx=0 tx=0).stop 2024-04-21T10:31:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.376+0000 7ff1864fd640 1 -- 172.21.15.26:0/137748768 shutdown_connections 2024-04-21T10:31:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.376+0000 7ff1864fd640 1 --2- 172.21.15.26:0/137748768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18010c5a0 0x7ff18010e9d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.376+0000 7ff1864fd640 1 --2- 172.21.15.26:0/137748768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff180103af0 0x7ff18010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.376+0000 7ff1864fd640 1 --2- 172.21.15.26:0/137748768 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1801031d0 0x7ff1801035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.376+0000 7ff1864fd640 1 -- 172.21.15.26:0/137748768 >> 172.21.15.26:0/137748768 conn(0x7ff1800771d0 msgr2=0x7ff1800775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.376+0000 7ff1864fd640 1 -- 172.21.15.26:0/137748768 shutdown_connections 2024-04-21T10:31:03.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.376+0000 7ff1864fd640 1 -- 172.21.15.26:0/137748768 wait complete. 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.376+0000 7ff1864fd640 1 Processor -- start 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.376+0000 7ff1864fd640 1 -- start start 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff1864fd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1801031d0 0x7ff1801986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff1864fd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180103af0 0x7ff180198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff1864fd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18010c5a0 0x7ff18019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff1864fd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff1801121c0 con 0x7ff18010c5a0 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff1864fd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff180112040 con 0x7ff1801031d0 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff1864fd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff180112340 con 0x7ff180103af0 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff184a9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18010c5a0 0x7ff18019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff184a9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18010c5a0 0x7ff18019cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46408/0 (socket says 172.21.15.26:46408) 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff184a9a640 1 -- 172.21.15.26:0/3683789680 learned_addr learned my addr 172.21.15.26:0/3683789680 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff17f7fe640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180103af0 0x7ff180198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:03.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff17ffff640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1801031d0 0x7ff1801986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff184a9a640 1 -- 172.21.15.26:0/3683789680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180103af0 msgr2=0x7ff180198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff184a9a640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180103af0 0x7ff180198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff184a9a640 1 -- 172.21.15.26:0/3683789680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1801031d0 msgr2=0x7ff1801986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff184a9a640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1801031d0 0x7ff1801986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff184a9a640 1 -- 172.21.15.26:0/3683789680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18019d6d0 con 0x7ff18010c5a0 2024-04-21T10:31:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.377+0000 7ff184a9a640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18010c5a0 0x7ff18019cfa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff17802c790 tx=0x7ff178003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.378+0000 7ff17d7fa640 1 -- 172.21.15.26:0/3683789680 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178039040 con 0x7ff18010c5a0 2024-04-21T10:31:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.378+0000 7ff1864fd640 1 -- 172.21.15.26:0/3683789680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff18019d960 con 0x7ff18010c5a0 2024-04-21T10:31:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.378+0000 7ff17d7fa640 1 -- 172.21.15.26:0/3683789680 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff178044070 con 0x7ff18010c5a0 2024-04-21T10:31:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.378+0000 7ff17d7fa640 1 -- 172.21.15.26:0/3683789680 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1780327c0 con 0x7ff18010c5a0 2024-04-21T10:31:03.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.378+0000 7ff1864fd640 1 -- 172.21.15.26:0/3683789680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1801a5250 con 0x7ff18010c5a0 2024-04-21T10:31:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.379+0000 7ff1864fd640 1 -- 172.21.15.26:0/3683789680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff180069230 con 0x7ff18010c5a0 2024-04-21T10:31:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.380+0000 7ff17d7fa640 1 -- 172.21.15.26:0/3683789680 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7ff178004040 con 0x7ff18010c5a0 2024-04-21T10:31:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.380+0000 7ff17d7fa640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff16c0a3ed0 0x7ff16c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.381+0000 7ff17ffff640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff16c0a3ed0 0x7ff16c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.381+0000 7ff17d7fa640 1 -- 172.21.15.26:0/3683789680 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7ff1780f2e80 con 0x7ff18010c5a0 2024-04-21T10:31:03.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.381+0000 7ff17ffff640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff16c0a3ed0 0x7ff16c0a6390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff168009aa0 tx=0x7ff168005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:03.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.385+0000 7ff17d7fa640 1 -- 172.21.15.26:0/3683789680 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1780bf400 con 0x7ff18010c5a0 2024-04-21T10:31:03.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.568+0000 7ff1864fd640 1 -- 172.21.15.26:0/3683789680 --> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1801076c0 con 0x7ff16c0a3ed0 2024-04-21T10:31:03.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.576+0000 7ff17d7fa640 1 -- 172.21.15.26:0/3683789680 <== mgr.24869 v2:172.21.15.26:6800/4245046040 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257737 (secure 0 0 0) 0x7ff1801076c0 con 0x7ff16c0a3ed0 2024-04-21T10:31:03.579 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:03.581 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-21T10:31:03.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.579+0000 7ff1864fd640 1 -- 172.21.15.26:0/3683789680 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff16c0a3ed0 msgr2=0x7ff16c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.579+0000 7ff1864fd640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff16c0a3ed0 0x7ff16c0a6390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff168009aa0 tx=0x7ff168005f90 comp rx=0 tx=0).stop 2024-04-21T10:31:03.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.579+0000 7ff1864fd640 1 -- 172.21.15.26:0/3683789680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18010c5a0 msgr2=0x7ff18019cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.579+0000 7ff1864fd640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18010c5a0 0x7ff18019cfa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff17802c790 tx=0x7ff178003ba0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.579+0000 7ff1864fd640 1 -- 172.21.15.26:0/3683789680 shutdown_connections 2024-04-21T10:31:03.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.579+0000 7ff1864fd640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff16c0a3ed0 0x7ff16c0a6390 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.579+0000 7ff1864fd640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff18010c5a0 0x7ff18019cfa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.579+0000 7ff1864fd640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff180103af0 0x7ff180198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.579+0000 7ff1864fd640 1 --2- 172.21.15.26:0/3683789680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1801031d0 0x7ff1801986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.579+0000 7ff1864fd640 1 -- 172.21.15.26:0/3683789680 >> 172.21.15.26:0/3683789680 conn(0x7ff1800771d0 msgr2=0x7ff1800fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:03.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.579+0000 7ff1864fd640 1 -- 172.21.15.26:0/3683789680 shutdown_connections 2024-04-21T10:31:03.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.579+0000 7ff1864fd640 1 -- 172.21.15.26:0/3683789680 wait complete. 2024-04-21T10:31:03.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: Updating smithi026:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:31:03.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:31:03.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:31:03.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:31:03.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:31:03.589 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: Updating smithi026:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/4103551741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.590 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:31:03.591 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:31:03.591 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:31:03.591 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.591 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3323333739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:31:03.591 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3094969291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:31:03.591 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:03 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3453980540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:31:03.592 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-21T10:31:01.819676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187859,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":937,"num_read_kb":627,"num_write":469,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364320,"kb_used_data":15132,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124577504,"statfs":{"total":1151940427776,"available":1151567364096,"internally_reserved":0,"allocated":15495168,"data_stored":9097425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.989497"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869187+0000","last_change":"2024-04-21T10:29:01.940996+0000","last_active":"2024-04-21T10:30:57.869187+0000","last_peered":"2024-04-21T10:30:57.869187+0000","last_clean":"2024-04-21T10:30:57.869187+0000","last_became_active":"2024-04-21T10:29:01.940662+0000","last_became_peered":"2024-04-21T10:29:01.940662+0000","last_unstale":"2024-04-21T10:30:57.869187+0000","last_undegraded":"2024-04-21T10:30:57.869187+0000","last_fullsized":"2024-04-21T10:30:57.869187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:16:01.434895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868745+0000","last_change":"2024-04-21T10:29:01.939238+0000","last_active":"2024-04-21T10:30:57.868745+0000","last_peered":"2024-04-21T10:30:57.868745+0000","last_clean":"2024-04-21T10:30:57.868745+0000","last_became_active":"2024-04-21T10:29:01.938907+0000","last_became_peered":"2024-04-21T10:29:01.938907+0000","last_unstale":"2024-04-21T10:30:57.868745+0000","last_undegraded":"2024-04-21T10:30:57.868745+0000","last_fullsized":"2024-04-21T10:30:57.868745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:09.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869188+0000","last_change":"2024-04-21T10:29:01.932338+0000","last_active":"2024-04-21T10:30:57.869188+0000","last_peered":"2024-04-21T10:30:57.869188+0000","last_clean":"2024-04-21T10:30:57.869188+0000","last_became_active":"2024-04-21T10:29:01.931916+0000","last_became_peered":"2024-04-21T10:29:01.931916+0000","last_unstale":"2024-04-21T10:30:57.869188+0000","last_undegraded":"2024-04-21T10:30:57.869188+0000","last_fullsized":"2024-04-21T10:30:57.869188+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:52.859695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869822+0000","last_change":"2024-04-21T10:29:01.945572+0000","last_active":"2024-04-21T10:30:57.869822+0000","last_peered":"2024-04-21T10:30:57.869822+0000","last_clean":"2024-04-21T10:30:57.869822+0000","last_became_active":"2024-04-21T10:29:01.940114+0000","last_became_peered":"2024-04-21T10:29:01.940114+0000","last_unstale":"2024-04-21T10:30:57.869822+0000","last_undegraded":"2024-04-21T10:30:57.869822+0000","last_fullsized":"2024-04-21T10:30:57.869822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:21.935573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868608+0000","last_change":"2024-04-21T10:29:01.946021+0000","last_active":"2024-04-21T10:30:57.868608+0000","last_peered":"2024-04-21T10:30:57.868608+0000","last_clean":"2024-04-21T10:30:57.868608+0000","last_became_active":"2024-04-21T10:29:01.945637+0000","last_became_peered":"2024-04-21T10:29:01.945637+0000","last_unstale":"2024-04-21T10:30:57.868608+0000","last_undegraded":"2024-04-21T10:30:57.868608+0000","last_fullsized":"2024-04-21T10:30:57.868608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:20.840494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868714+0000","last_change":"2024-04-21T10:29:01.935402+0000","last_active":"2024-04-21T10:30:57.868714+0000","last_peered":"2024-04-21T10:30:57.868714+0000","last_clean":"2024-04-21T10:30:57.868714+0000","last_became_active":"2024-04-21T10:29:01.934994+0000","last_became_peered":"2024-04-21T10:29:01.934994+0000","last_unstale":"2024-04-21T10:30:57.868714+0000","last_undegraded":"2024-04-21T10:30:57.868714+0000","last_fullsized":"2024-04-21T10:30:57.868714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:18.320530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868977+0000","last_change":"2024-04-21T10:29:01.943185+0000","last_active":"2024-04-21T10:30:57.868977+0000","last_peered":"2024-04-21T10:30:57.868977+0000","last_clean":"2024-04-21T10:30:57.868977+0000","last_became_active":"2024-04-21T10:29:01.942805+0000","last_became_peered":"2024-04-21T10:29:01.942805+0000","last_unstale":"2024-04-21T10:30:57.868977+0000","last_undegraded":"2024-04-21T10:30:57.868977+0000","last_fullsized":"2024-04-21T10:30:57.868977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:06.776472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868710+0000","last_change":"2024-04-21T10:29:01.940995+0000","last_active":"2024-04-21T10:30:57.868710+0000","last_peered":"2024-04-21T10:30:57.868710+0000","last_clean":"2024-04-21T10:30:57.868710+0000","last_became_active":"2024-04-21T10:29:01.939173+0000","last_became_peered":"2024-04-21T10:29:01.939173+0000","last_unstale":"2024-04-21T10:30:57.868710+0000","last_undegraded":"2024-04-21T10:30:57.868710+0000","last_fullsized":"2024-04-21T10:30:57.868710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.740147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807868+0000","last_change":"2024-04-21T10:29:01.944284+0000","last_active":"2024-04-21T10:30:57.807868+0000","last_peered":"2024-04-21T10:30:57.807868+0000","last_clean":"2024-04-21T10:30:57.807868+0000","last_became_active":"2024-04-21T10:29:01.943965+0000","last_became_peered":"2024-04-21T10:29:01.943965+0000","last_unstale":"2024-04-21T10:30:57.807868+0000","last_undegraded":"2024-04-21T10:30:57.807868+0000","last_fullsized":"2024-04-21T10:30:57.807868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:08:54.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869273+0000","last_change":"2024-04-21T10:29:01.936250+0000","last_active":"2024-04-21T10:30:57.869273+0000","last_peered":"2024-04-21T10:30:57.869273+0000","last_clean":"2024-04-21T10:30:57.869273+0000","last_became_active":"2024-04-21T10:29:01.935904+0000","last_became_peered":"2024-04-21T10:29:01.935904+0000","last_unstale":"2024-04-21T10:30:57.869273+0000","last_undegraded":"2024-04-21T10:30:57.869273+0000","last_fullsized":"2024-04-21T10:30:57.869273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:35.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869510+0000","last_change":"2024-04-21T10:29:01.946550+0000","last_active":"2024-04-21T10:30:57.869510+0000","last_peered":"2024-04-21T10:30:57.869510+0000","last_clean":"2024-04-21T10:30:57.869510+0000","last_became_active":"2024-04-21T10:29:01.946170+0000","last_became_peered":"2024-04-21T10:29:01.946170+0000","last_unstale":"2024-04-21T10:30:57.869510+0000","last_undegraded":"2024-04-21T10:30:57.869510+0000","last_fullsized":"2024-04-21T10:30:57.869510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:25:47.671821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869011+0000","last_change":"2024-04-21T10:29:01.946036+0000","last_active":"2024-04-21T10:30:57.869011+0000","last_peered":"2024-04-21T10:30:57.869011+0000","last_clean":"2024-04-21T10:30:57.869011+0000","last_became_active":"2024-04-21T10:29:01.940705+0000","last_became_peered":"2024-04-21T10:29:01.940705+0000","last_unstale":"2024-04-21T10:30:57.869011+0000","last_undegraded":"2024-04-21T10:30:57.869011+0000","last_fullsized":"2024-04-21T10:30:57.869011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:11:33.196059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805947+0000","last_change":"2024-04-21T10:29:01.940438+0000","last_active":"2024-04-21T10:30:57.805947+0000","last_peered":"2024-04-21T10:30:57.805947+0000","last_clean":"2024-04-21T10:30:57.805947+0000","last_became_active":"2024-04-21T10:29:01.940085+0000","last_became_peered":"2024-04-21T10:29:01.940085+0000","last_unstale":"2024-04-21T10:30:57.805947+0000","last_undegraded":"2024-04-21T10:30:57.805947+0000","last_fullsized":"2024-04-21T10:30:57.805947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:22:47.828858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868102+0000","last_change":"2024-04-21T10:29:01.941667+0000","last_active":"2024-04-21T10:30:57.868102+0000","last_peered":"2024-04-21T10:30:57.868102+0000","last_clean":"2024-04-21T10:30:57.868102+0000","last_became_active":"2024-04-21T10:29:01.940745+0000","last_became_peered":"2024-04-21T10:29:01.940745+0000","last_unstale":"2024-04-21T10:30:57.868102+0000","last_undegraded":"2024-04-21T10:30:57.868102+0000","last_fullsized":"2024-04-21T10:30:57.868102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:45:51.825780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.867997+0000","last_change":"2024-04-21T10:29:01.947771+0000","last_active":"2024-04-21T10:30:57.867997+0000","last_peered":"2024-04-21T10:30:57.867997+0000","last_clean":"2024-04-21T10:30:57.867997+0000","last_became_active":"2024-04-21T10:29:01.946818+0000","last_became_peered":"2024-04-21T10:29:01.946818+0000","last_unstale":"2024-04-21T10:30:57.867997+0000","last_undegraded":"2024-04-21T10:30:57.867997+0000","last_fullsized":"2024-04-21T10:30:57.867997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:18:42.159152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807344+0000","last_change":"2024-04-21T10:29:01.942499+0000","last_active":"2024-04-21T10:30:57.807344+0000","last_peered":"2024-04-21T10:30:57.807344+0000","last_clean":"2024-04-21T10:30:57.807344+0000","last_became_active":"2024-04-21T10:29:01.942136+0000","last_became_peered":"2024-04-21T10:29:01.942136+0000","last_unstale":"2024-04-21T10:30:57.807344+0000","last_undegraded":"2024-04-21T10:30:57.807344+0000","last_fullsized":"2024-04-21T10:30:57.807344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:34:11.822230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869190+0000","last_change":"2024-04-21T10:29:01.948822+0000","last_active":"2024-04-21T10:30:57.869190+0000","last_peered":"2024-04-21T10:30:57.869190+0000","last_clean":"2024-04-21T10:30:57.869190+0000","last_became_active":"2024-04-21T10:29:01.948492+0000","last_became_peered":"2024-04-21T10:29:01.948492+0000","last_unstale":"2024-04-21T10:30:57.869190+0000","last_undegraded":"2024-04-21T10:30:57.869190+0000","last_fullsized":"2024-04-21T10:30:57.869190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:06.992585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869064+0000","last_change":"2024-04-21T10:29:01.931265+0000","last_active":"2024-04-21T10:30:57.869064+0000","last_peered":"2024-04-21T10:30:57.869064+0000","last_clean":"2024-04-21T10:30:57.869064+0000","last_became_active":"2024-04-21T10:29:01.930877+0000","last_became_peered":"2024-04-21T10:29:01.930877+0000","last_unstale":"2024-04-21T10:30:57.869064+0000","last_undegraded":"2024-04-21T10:30:57.869064+0000","last_fullsized":"2024-04-21T10:30:57.869064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:49:00.951226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869590+0000","last_change":"2024-04-21T10:29:01.941718+0000","last_active":"2024-04-21T10:30:57.869590+0000","last_peered":"2024-04-21T10:30:57.869590+0000","last_clean":"2024-04-21T10:30:57.869590+0000","last_became_active":"2024-04-21T10:29:01.941386+0000","last_became_peered":"2024-04-21T10:29:01.941386+0000","last_unstale":"2024-04-21T10:30:57.869590+0000","last_undegraded":"2024-04-21T10:30:57.869590+0000","last_fullsized":"2024-04-21T10:30:57.869590+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:51:29.847961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869350+0000","last_change":"2024-04-21T10:29:00.923670+0000","last_active":"2024-04-21T10:30:57.869350+0000","last_peered":"2024-04-21T10:30:57.869350+0000","last_clean":"2024-04-21T10:30:57.869350+0000","last_became_active":"2024-04-21T10:29:00.923100+0000","last_became_peered":"2024-04-21T10:29:00.923100+0000","last_unstale":"2024-04-21T10:30:57.869350+0000","last_undegraded":"2024-04-21T10:30:57.869350+0000","last_fullsized":"2024-04-21T10:30:57.869350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:45.692961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869383+0000","last_change":"2024-04-21T10:29:01.938131+0000","last_active":"2024-04-21T10:30:57.869383+0000","last_peered":"2024-04-21T10:30:57.869383+0000","last_clean":"2024-04-21T10:30:57.869383+0000","last_became_active":"2024-04-21T10:29:01.937752+0000","last_became_peered":"2024-04-21T10:29:01.937752+0000","last_unstale":"2024-04-21T10:30:57.869383+0000","last_undegraded":"2024-04-21T10:30:57.869383+0000","last_fullsized":"2024-04-21T10:30:57.869383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:06:49.383554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869489+0000","last_change":"2024-04-21T10:29:00.929877+0000","last_active":"2024-04-21T10:30:57.869489+0000","last_peered":"2024-04-21T10:30:57.869489+0000","last_clean":"2024-04-21T10:30:57.869489+0000","last_became_active":"2024-04-21T10:29:00.929564+0000","last_became_peered":"2024-04-21T10:29:00.929564+0000","last_unstale":"2024-04-21T10:30:57.869489+0000","last_undegraded":"2024-04-21T10:30:57.869489+0000","last_fullsized":"2024-04-21T10:30:57.869489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:35.617533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869209+0000","last_change":"2024-04-21T10:29:01.942782+0000","last_active":"2024-04-21T10:30:57.869209+0000","last_peered":"2024-04-21T10:30:57.869209+0000","last_clean":"2024-04-21T10:30:57.869209+0000","last_became_active":"2024-04-21T10:29:01.940501+0000","last_became_peered":"2024-04-21T10:29:01.940501+0000","last_unstale":"2024-04-21T10:30:57.869209+0000","last_undegraded":"2024-04-21T10:30:57.869209+0000","last_fullsized":"2024-04-21T10:30:57.869209+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:04:03.431016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869465+0000","last_change":"2024-04-21T10:29:00.924395+0000","last_active":"2024-04-21T10:30:57.869465+0000","last_peered":"2024-04-21T10:30:57.869465+0000","last_clean":"2024-04-21T10:30:57.869465+0000","last_became_active":"2024-04-21T10:29:00.924045+0000","last_became_peered":"2024-04-21T10:29:00.924045+0000","last_unstale":"2024-04-21T10:30:57.869465+0000","last_undegraded":"2024-04-21T10:30:57.869465+0000","last_fullsized":"2024-04-21T10:30:57.869465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:08:53.922213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807373+0000","last_change":"2024-04-21T10:29:01.934508+0000","last_active":"2024-04-21T10:30:57.807373+0000","last_peered":"2024-04-21T10:30:57.807373+0000","last_clean":"2024-04-21T10:30:57.807373+0000","last_became_active":"2024-04-21T10:29:01.934051+0000","last_became_peered":"2024-04-21T10:29:01.934051+0000","last_unstale":"2024-04-21T10:30:57.807373+0000","last_undegraded":"2024-04-21T10:30:57.807373+0000","last_fullsized":"2024-04-21T10:30:57.807373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:49:27.686603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868924+0000","last_change":"2024-04-21T10:29:00.925850+0000","last_active":"2024-04-21T10:30:57.868924+0000","last_peered":"2024-04-21T10:30:57.868924+0000","last_clean":"2024-04-21T10:30:57.868924+0000","last_became_active":"2024-04-21T10:29:00.925521+0000","last_became_peered":"2024-04-21T10:29:00.925521+0000","last_unstale":"2024-04-21T10:30:57.868924+0000","last_undegraded":"2024-04-21T10:30:57.868924+0000","last_fullsized":"2024-04-21T10:30:57.868924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:36:55.403413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807733+0000","last_change":"2024-04-21T10:29:01.930046+0000","last_active":"2024-04-21T10:30:57.807733+0000","last_peered":"2024-04-21T10:30:57.807733+0000","last_clean":"2024-04-21T10:30:57.807733+0000","last_became_active":"2024-04-21T10:29:01.929736+0000","last_became_peered":"2024-04-21T10:29:01.929736+0000","last_unstale":"2024-04-21T10:30:57.807733+0000","last_undegraded":"2024-04-21T10:30:57.807733+0000","last_fullsized":"2024-04-21T10:30:57.807733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:11:10.833267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":143,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.047873+0000","last_change":"2024-04-21T10:29:00.927936+0000","last_active":"2024-04-21T10:30:58.047873+0000","last_peered":"2024-04-21T10:30:58.047873+0000","last_clean":"2024-04-21T10:30:58.047873+0000","last_became_active":"2024-04-21T10:29:00.927352+0000","last_became_peered":"2024-04-21T10:29:00.927352+0000","last_unstale":"2024-04-21T10:30:58.047873+0000","last_undegraded":"2024-04-21T10:30:58.047873+0000","last_fullsized":"2024-04-21T10:30:58.047873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:14:10.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869207+0000","last_change":"2024-04-21T10:30:03.306538+0000","last_active":"2024-04-21T10:30:57.869207+0000","last_peered":"2024-04-21T10:30:57.869207+0000","last_clean":"2024-04-21T10:30:57.869207+0000","last_became_active":"2024-04-21T10:30:03.306197+0000","last_became_peered":"2024-04-21T10:30:03.306197+0000","last_unstale":"2024-04-21T10:30:57.869207+0000","last_undegraded":"2024-04-21T10:30:57.869207+0000","last_fullsized":"2024-04-21T10:30:57.869207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:28:25.688821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868668+0000","last_change":"2024-04-21T10:30:03.522106+0000","last_active":"2024-04-21T10:30:57.868668+0000","last_peered":"2024-04-21T10:30:57.868668+0000","last_clean":"2024-04-21T10:30:57.868668+0000","last_became_active":"2024-04-21T10:30:03.519189+0000","last_became_peered":"2024-04-21T10:30:03.519189+0000","last_unstale":"2024-04-21T10:30:57.868668+0000","last_undegraded":"2024-04-21T10:30:57.868668+0000","last_fullsized":"2024-04-21T10:30:57.868668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:49:39.930818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.808006+0000","last_change":"2024-04-21T10:29:01.918627+0000","last_active":"2024-04-21T10:30:57.808006+0000","last_peered":"2024-04-21T10:30:57.808006+0000","last_clean":"2024-04-21T10:30:57.808006+0000","last_became_active":"2024-04-21T10:29:01.917993+0000","last_became_peered":"2024-04-21T10:29:01.917993+0000","last_unstale":"2024-04-21T10:30:57.808006+0000","last_undegraded":"2024-04-21T10:30:57.808006+0000","last_fullsized":"2024-04-21T10:30:57.808006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:58.891918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805799+0000","last_change":"2024-04-21T10:29:00.923613+0000","last_active":"2024-04-21T10:30:57.805799+0000","last_peered":"2024-04-21T10:30:57.805799+0000","last_clean":"2024-04-21T10:30:57.805799+0000","last_became_active":"2024-04-21T10:29:00.923001+0000","last_became_peered":"2024-04-21T10:29:00.923001+0000","last_unstale":"2024-04-21T10:30:57.805799+0000","last_undegraded":"2024-04-21T10:30:57.805799+0000","last_fullsized":"2024-04-21T10:30:57.805799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:44:53.593903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868537+0000","last_change":"2024-04-21T10:29:01.934974+0000","last_active":"2024-04-21T10:30:57.868537+0000","last_peered":"2024-04-21T10:30:57.868537+0000","last_clean":"2024-04-21T10:30:57.868537+0000","last_became_active":"2024-04-21T10:29:01.932443+0000","last_became_peered":"2024-04-21T10:29:01.932443+0000","last_unstale":"2024-04-21T10:30:57.868537+0000","last_undegraded":"2024-04-21T10:30:57.868537+0000","last_fullsized":"2024-04-21T10:30:57.868537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:24:47.190142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807475+0000","last_change":"2024-04-21T10:29:00.928179+0000","last_active":"2024-04-21T10:30:57.807475+0000","last_peered":"2024-04-21T10:30:57.807475+0000","last_clean":"2024-04-21T10:30:57.807475+0000","last_became_active":"2024-04-21T10:29:00.927588+0000","last_became_peered":"2024-04-21T10:29:00.927588+0000","last_unstale":"2024-04-21T10:30:57.807475+0000","last_undegraded":"2024-04-21T10:30:57.807475+0000","last_fullsized":"2024-04-21T10:30:57.807475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:04:41.081424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868724+0000","last_change":"2024-04-21T10:29:01.944421+0000","last_active":"2024-04-21T10:30:57.868724+0000","last_peered":"2024-04-21T10:30:57.868724+0000","last_clean":"2024-04-21T10:30:57.868724+0000","last_became_active":"2024-04-21T10:29:01.943572+0000","last_became_peered":"2024-04-21T10:29:01.943572+0000","last_unstale":"2024-04-21T10:30:57.868724+0000","last_undegraded":"2024-04-21T10:30:57.868724+0000","last_fullsized":"2024-04-21T10:30:57.868724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:34:01.690600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":138,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.973545+0000","last_change":"2024-04-21T10:29:00.924406+0000","last_active":"2024-04-21T10:30:57.973545+0000","last_peered":"2024-04-21T10:30:57.973545+0000","last_clean":"2024-04-21T10:30:57.973545+0000","last_became_active":"2024-04-21T10:29:00.924094+0000","last_became_peered":"2024-04-21T10:29:00.924094+0000","last_unstale":"2024-04-21T10:30:57.973545+0000","last_undegraded":"2024-04-21T10:30:57.973545+0000","last_fullsized":"2024-04-21T10:30:57.973545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:44.673751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807739+0000","last_change":"2024-04-21T10:29:01.933048+0000","last_active":"2024-04-21T10:30:57.807739+0000","last_peered":"2024-04-21T10:30:57.807739+0000","last_clean":"2024-04-21T10:30:57.807739+0000","last_became_active":"2024-04-21T10:29:01.932659+0000","last_became_peered":"2024-04-21T10:29:01.932659+0000","last_unstale":"2024-04-21T10:30:57.807739+0000","last_undegraded":"2024-04-21T10:30:57.807739+0000","last_fullsized":"2024-04-21T10:30:57.807739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:22:13.712112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869356+0000","last_change":"2024-04-21T10:29:00.923020+0000","last_active":"2024-04-21T10:30:57.869356+0000","last_peered":"2024-04-21T10:30:57.869356+0000","last_clean":"2024-04-21T10:30:57.869356+0000","last_became_active":"2024-04-21T10:29:00.922678+0000","last_became_peered":"2024-04-21T10:29:00.922678+0000","last_unstale":"2024-04-21T10:30:57.869356+0000","last_undegraded":"2024-04-21T10:30:57.869356+0000","last_fullsized":"2024-04-21T10:30:57.869356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:13:34.698082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869351+0000","last_change":"2024-04-21T10:29:01.947833+0000","last_active":"2024-04-21T10:30:57.869351+0000","last_peered":"2024-04-21T10:30:57.869351+0000","last_clean":"2024-04-21T10:30:57.869351+0000","last_became_active":"2024-04-21T10:29:01.947351+0000","last_became_peered":"2024-04-21T10:29:01.947351+0000","last_unstale":"2024-04-21T10:30:57.869351+0000","last_undegraded":"2024-04-21T10:30:57.869351+0000","last_fullsized":"2024-04-21T10:30:57.869351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:54.538731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807854+0000","last_change":"2024-04-21T10:29:00.932309+0000","last_active":"2024-04-21T10:30:57.807854+0000","last_peered":"2024-04-21T10:30:57.807854+0000","last_clean":"2024-04-21T10:30:57.807854+0000","last_became_active":"2024-04-21T10:29:00.931935+0000","last_became_peered":"2024-04-21T10:29:00.931935+0000","last_unstale":"2024-04-21T10:30:57.807854+0000","last_undegraded":"2024-04-21T10:30:57.807854+0000","last_fullsized":"2024-04-21T10:30:57.807854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:37:37.604381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807591+0000","last_change":"2024-04-21T10:29:01.946543+0000","last_active":"2024-04-21T10:30:57.807591+0000","last_peered":"2024-04-21T10:30:57.807591+0000","last_clean":"2024-04-21T10:30:57.807591+0000","last_became_active":"2024-04-21T10:29:01.946210+0000","last_became_peered":"2024-04-21T10:29:01.946210+0000","last_unstale":"2024-04-21T10:30:57.807591+0000","last_undegraded":"2024-04-21T10:30:57.807591+0000","last_fullsized":"2024-04-21T10:30:57.807591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:49.368851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'38","reported_seq":297,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.050618+0000","last_change":"2024-04-21T10:29:00.924109+0000","last_active":"2024-04-21T10:30:58.050618+0000","last_peered":"2024-04-21T10:30:58.050618+0000","last_clean":"2024-04-21T10:30:58.050618+0000","last_became_active":"2024-04-21T10:29:00.923735+0000","last_became_peered":"2024-04-21T10:29:00.923735+0000","last_unstale":"2024-04-21T10:30:58.050618+0000","last_undegraded":"2024-04-21T10:30:58.050618+0000","last_fullsized":"2024-04-21T10:30:58.050618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:13:54.015774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66735,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":238,"num_read_kb":184,"num_write":39,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869133+0000","last_change":"2024-04-21T10:29:01.937211+0000","last_active":"2024-04-21T10:30:57.869133+0000","last_peered":"2024-04-21T10:30:57.869133+0000","last_clean":"2024-04-21T10:30:57.869133+0000","last_became_active":"2024-04-21T10:29:01.936891+0000","last_became_peered":"2024-04-21T10:29:01.936891+0000","last_unstale":"2024-04-21T10:30:57.869133+0000","last_undegraded":"2024-04-21T10:30:57.869133+0000","last_fullsized":"2024-04-21T10:30:57.869133+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:26.062236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869231+0000","last_change":"2024-04-21T10:29:00.920626+0000","last_active":"2024-04-21T10:30:57.869231+0000","last_peered":"2024-04-21T10:30:57.869231+0000","last_clean":"2024-04-21T10:30:57.869231+0000","last_became_active":"2024-04-21T10:29:00.920317+0000","last_became_peered":"2024-04-21T10:29:00.920317+0000","last_unstale":"2024-04-21T10:30:57.869231+0000","last_undegraded":"2024-04-21T10:30:57.869231+0000","last_fullsized":"2024-04-21T10:30:57.869231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:41:18.373403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805778+0000","last_change":"2024-04-21T10:29:01.942494+0000","last_active":"2024-04-21T10:30:57.805778+0000","last_peered":"2024-04-21T10:30:57.805778+0000","last_clean":"2024-04-21T10:30:57.805778+0000","last_became_active":"2024-04-21T10:29:01.942151+0000","last_became_peered":"2024-04-21T10:29:01.942151+0000","last_unstale":"2024-04-21T10:30:57.805778+0000","last_undegraded":"2024-04-21T10:30:57.805778+0000","last_fullsized":"2024-04-21T10:30:57.805778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:39.071158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807798+0000","last_change":"2024-04-21T10:29:00.928221+0000","last_active":"2024-04-21T10:30:57.807798+0000","last_peered":"2024-04-21T10:30:57.807798+0000","last_clean":"2024-04-21T10:30:57.807798+0000","last_became_active":"2024-04-21T10:29:00.927873+0000","last_became_peered":"2024-04-21T10:29:00.927873+0000","last_unstale":"2024-04-21T10:30:57.807798+0000","last_undegraded":"2024-04-21T10:30:57.807798+0000","last_fullsized":"2024-04-21T10:30:57.807798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:14.898456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869676+0000","last_change":"2024-04-21T10:29:01.930375+0000","last_active":"2024-04-21T10:30:57.869676+0000","last_peered":"2024-04-21T10:30:57.869676+0000","last_clean":"2024-04-21T10:30:57.869676+0000","last_became_active":"2024-04-21T10:29:01.930009+0000","last_became_peered":"2024-04-21T10:29:01.930009+0000","last_unstale":"2024-04-21T10:30:57.869676+0000","last_undegraded":"2024-04-21T10:30:57.869676+0000","last_fullsized":"2024-04-21T10:30:57.869676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:34.111797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.053554+0000","last_change":"2024-04-21T10:29:00.922981+0000","last_active":"2024-04-21T10:30:58.053554+0000","last_peered":"2024-04-21T10:30:58.053554+0000","last_clean":"2024-04-21T10:30:58.053554+0000","last_became_active":"2024-04-21T10:29:00.922668+0000","last_became_peered":"2024-04-21T10:29:00.922668+0000","last_unstale":"2024-04-21T10:30:58.053554+0000","last_undegraded":"2024-04-21T10:30:58.053554+0000","last_fullsized":"2024-04-21T10:30:58.053554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:47.109375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868377+0000","last_change":"2024-04-21T10:29:01.942530+0000","last_active":"2024-04-21T10:30:57.868377+0000","last_peered":"2024-04-21T10:30:57.868377+0000","last_clean":"2024-04-21T10:30:57.868377+0000","last_became_active":"2024-04-21T10:29:01.942162+0000","last_became_peered":"2024-04-21T10:29:01.942162+0000","last_unstale":"2024-04-21T10:30:57.868377+0000","last_undegraded":"2024-04-21T10:30:57.868377+0000","last_fullsized":"2024-04-21T10:30:57.868377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:46:19.138930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868297+0000","last_change":"2024-04-21T10:30:03.544712+0000","last_active":"2024-04-21T10:30:57.868297+0000","last_peered":"2024-04-21T10:30:57.868297+0000","last_clean":"2024-04-21T10:30:57.868297+0000","last_became_active":"2024-04-21T10:30:03.519730+0000","last_became_peered":"2024-04-21T10:30:03.519730+0000","last_unstale":"2024-04-21T10:30:57.868297+0000","last_undegraded":"2024-04-21T10:30:57.868297+0000","last_fullsized":"2024-04-21T10:30:57.868297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:54:33.184653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807494+0000","last_change":"2024-04-21T10:29:01.929696+0000","last_active":"2024-04-21T10:30:57.807494+0000","last_peered":"2024-04-21T10:30:57.807494+0000","last_clean":"2024-04-21T10:30:57.807494+0000","last_became_active":"2024-04-21T10:29:01.929328+0000","last_became_peered":"2024-04-21T10:29:01.929328+0000","last_unstale":"2024-04-21T10:30:57.807494+0000","last_undegraded":"2024-04-21T10:30:57.807494+0000","last_fullsized":"2024-04-21T10:30:57.807494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:55:42.735595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869457+0000","last_change":"2024-04-21T10:29:00.925981+0000","last_active":"2024-04-21T10:30:57.869457+0000","last_peered":"2024-04-21T10:30:57.869457+0000","last_clean":"2024-04-21T10:30:57.869457+0000","last_became_active":"2024-04-21T10:29:00.925560+0000","last_became_peered":"2024-04-21T10:29:00.925560+0000","last_unstale":"2024-04-21T10:30:57.869457+0000","last_undegraded":"2024-04-21T10:30:57.869457+0000","last_fullsized":"2024-04-21T10:30:57.869457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:37.467522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868974+0000","last_change":"2024-04-21T10:29:01.941961+0000","last_active":"2024-04-21T10:30:57.868974+0000","last_peered":"2024-04-21T10:30:57.868974+0000","last_clean":"2024-04-21T10:30:57.868974+0000","last_became_active":"2024-04-21T10:29:01.941649+0000","last_became_peered":"2024-04-21T10:29:01.941649+0000","last_unstale":"2024-04-21T10:30:57.868974+0000","last_undegraded":"2024-04-21T10:30:57.868974+0000","last_fullsized":"2024-04-21T10:30:57.868974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:00:56.803856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869019+0000","last_change":"2024-04-21T10:29:00.929504+0000","last_active":"2024-04-21T10:30:57.869019+0000","last_peered":"2024-04-21T10:30:57.869019+0000","last_clean":"2024-04-21T10:30:57.869019+0000","last_became_active":"2024-04-21T10:29:00.929172+0000","last_became_peered":"2024-04-21T10:29:00.929172+0000","last_unstale":"2024-04-21T10:30:57.869019+0000","last_undegraded":"2024-04-21T10:30:57.869019+0000","last_fullsized":"2024-04-21T10:30:57.869019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:40.512175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869000+0000","last_change":"2024-04-21T10:29:01.939333+0000","last_active":"2024-04-21T10:30:57.869000+0000","last_peered":"2024-04-21T10:30:57.869000+0000","last_clean":"2024-04-21T10:30:57.869000+0000","last_became_active":"2024-04-21T10:29:01.938924+0000","last_became_peered":"2024-04-21T10:29:01.938924+0000","last_unstale":"2024-04-21T10:30:57.869000+0000","last_undegraded":"2024-04-21T10:30:57.869000+0000","last_fullsized":"2024-04-21T10:30:57.869000+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:20:01.287604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868977+0000","last_change":"2024-04-21T10:29:00.927455+0000","last_active":"2024-04-21T10:30:57.868977+0000","last_peered":"2024-04-21T10:30:57.868977+0000","last_clean":"2024-04-21T10:30:57.868977+0000","last_became_active":"2024-04-21T10:29:00.927180+0000","last_became_peered":"2024-04-21T10:29:00.927180+0000","last_unstale":"2024-04-21T10:30:57.868977+0000","last_undegraded":"2024-04-21T10:30:57.868977+0000","last_fullsized":"2024-04-21T10:30:57.868977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:11:57.166607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868482+0000","last_change":"2024-04-21T10:29:01.937646+0000","last_active":"2024-04-21T10:30:57.868482+0000","last_peered":"2024-04-21T10:30:57.868482+0000","last_clean":"2024-04-21T10:30:57.868482+0000","last_became_active":"2024-04-21T10:29:01.937216+0000","last_became_peered":"2024-04-21T10:29:01.937216+0000","last_unstale":"2024-04-21T10:30:57.868482+0000","last_undegraded":"2024-04-21T10:30:57.868482+0000","last_fullsized":"2024-04-21T10:30:57.868482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:38.265252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807620+0000","last_change":"2024-04-21T10:29:01.926860+0000","last_active":"2024-04-21T10:30:57.807620+0000","last_peered":"2024-04-21T10:30:57.807620+0000","last_clean":"2024-04-21T10:30:57.807620+0000","last_became_active":"2024-04-21T10:29:01.925951+0000","last_became_peered":"2024-04-21T10:29:01.925951+0000","last_unstale":"2024-04-21T10:30:57.807620+0000","last_undegraded":"2024-04-21T10:30:57.807620+0000","last_fullsized":"2024-04-21T10:30:57.807620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:01:55.127301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":140,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.973729+0000","last_change":"2024-04-21T10:29:00.926502+0000","last_active":"2024-04-21T10:30:57.973729+0000","last_peered":"2024-04-21T10:30:57.973729+0000","last_clean":"2024-04-21T10:30:57.973729+0000","last_became_active":"2024-04-21T10:29:00.926175+0000","last_became_peered":"2024-04-21T10:29:00.926175+0000","last_unstale":"2024-04-21T10:30:57.973729+0000","last_undegraded":"2024-04-21T10:30:57.973729+0000","last_fullsized":"2024-04-21T10:30:57.973729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:40:34.774085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868727+0000","last_change":"2024-04-21T10:30:03.519801+0000","last_active":"2024-04-21T10:30:57.868727+0000","last_peered":"2024-04-21T10:30:57.868727+0000","last_clean":"2024-04-21T10:30:57.868727+0000","last_became_active":"2024-04-21T10:30:03.519351+0000","last_became_peered":"2024-04-21T10:30:03.519351+0000","last_unstale":"2024-04-21T10:30:57.868727+0000","last_undegraded":"2024-04-21T10:30:57.868727+0000","last_fullsized":"2024-04-21T10:30:57.868727+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:13:36.675062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868748+0000","last_change":"2024-04-21T10:29:00.921748+0000","last_active":"2024-04-21T10:30:57.868748+0000","last_peered":"2024-04-21T10:30:57.868748+0000","last_clean":"2024-04-21T10:30:57.868748+0000","last_became_active":"2024-04-21T10:29:00.921408+0000","last_became_peered":"2024-04-21T10:29:00.921408+0000","last_unstale":"2024-04-21T10:30:57.868748+0000","last_undegraded":"2024-04-21T10:30:57.868748+0000","last_fullsized":"2024-04-21T10:30:57.868748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:52:27.606280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869101+0000","last_change":"2024-04-21T10:27:49.787009+0000","last_active":"2024-04-21T10:30:57.869101+0000","last_peered":"2024-04-21T10:30:57.869101+0000","last_clean":"2024-04-21T10:30:57.869101+0000","last_became_active":"2024-04-21T10:27:49.786530+0000","last_became_peered":"2024-04-21T10:27:49.786530+0000","last_unstale":"2024-04-21T10:30:57.869101+0000","last_undegraded":"2024-04-21T10:30:57.869101+0000","last_fullsized":"2024-04-21T10:30:57.869101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_clean_scrub_stamp":"2024-04-21T10:24:28.367927+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:20:12.106231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805642+0000","last_change":"2024-04-21T10:30:03.305720+0000","last_active":"2024-04-21T10:30:57.805642+0000","last_peered":"2024-04-21T10:30:57.805642+0000","last_clean":"2024-04-21T10:30:57.805642+0000","last_became_active":"2024-04-21T10:30:03.305368+0000","last_became_peered":"2024-04-21T10:30:03.305368+0000","last_unstale":"2024-04-21T10:30:57.805642+0000","last_undegraded":"2024-04-21T10:30:57.805642+0000","last_fullsized":"2024-04-21T10:30:57.805642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:16.668743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869177+0000","last_change":"2024-04-21T10:29:00.927521+0000","last_active":"2024-04-21T10:30:57.869177+0000","last_peered":"2024-04-21T10:30:57.869177+0000","last_clean":"2024-04-21T10:30:57.869177+0000","last_became_active":"2024-04-21T10:29:00.927202+0000","last_became_peered":"2024-04-21T10:29:00.927202+0000","last_unstale":"2024-04-21T10:30:57.869177+0000","last_undegraded":"2024-04-21T10:30:57.869177+0000","last_fullsized":"2024-04-21T10:30:57.869177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:11:32.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869508+0000","last_change":"2024-04-21T10:29:00.930728+0000","last_active":"2024-04-21T10:30:57.869508+0000","last_peered":"2024-04-21T10:30:57.869508+0000","last_clean":"2024-04-21T10:30:57.869508+0000","last_became_active":"2024-04-21T10:29:00.930457+0000","last_became_peered":"2024-04-21T10:29:00.930457+0000","last_unstale":"2024-04-21T10:30:57.869508+0000","last_undegraded":"2024-04-21T10:30:57.869508+0000","last_fullsized":"2024-04-21T10:30:57.869508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:28.032954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.806062+0000","last_change":"2024-04-21T10:29:01.932278+0000","last_active":"2024-04-21T10:30:57.806062+0000","last_peered":"2024-04-21T10:30:57.806062+0000","last_clean":"2024-04-21T10:30:57.806062+0000","last_became_active":"2024-04-21T10:29:01.931774+0000","last_became_peered":"2024-04-21T10:29:01.931774+0000","last_unstale":"2024-04-21T10:30:57.806062+0000","last_undegraded":"2024-04-21T10:30:57.806062+0000","last_fullsized":"2024-04-21T10:30:57.806062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:34:25.858475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'44","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.008537+0000","last_change":"2024-04-21T10:29:00.926071+0000","last_active":"2024-04-21T10:30:58.008537+0000","last_peered":"2024-04-21T10:30:58.008537+0000","last_clean":"2024-04-21T10:30:58.008537+0000","last_became_active":"2024-04-21T10:29:00.925607+0000","last_became_peered":"2024-04-21T10:29:00.925607+0000","last_unstale":"2024-04-21T10:30:58.008537+0000","last_undegraded":"2024-04-21T10:30:58.008537+0000","last_fullsized":"2024-04-21T10:30:58.008537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:26.350652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":78,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869595+0000","last_change":"2024-04-21T10:29:01.941365+0000","last_active":"2024-04-21T10:30:57.869595+0000","last_peered":"2024-04-21T10:30:57.869595+0000","last_clean":"2024-04-21T10:30:57.869595+0000","last_became_active":"2024-04-21T10:29:01.940102+0000","last_became_peered":"2024-04-21T10:29:01.940102+0000","last_unstale":"2024-04-21T10:30:57.869595+0000","last_undegraded":"2024-04-21T10:30:57.869595+0000","last_fullsized":"2024-04-21T10:30:57.869595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:21.976855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869277+0000","last_change":"2024-04-21T10:29:00.924322+0000","last_active":"2024-04-21T10:30:57.869277+0000","last_peered":"2024-04-21T10:30:57.869277+0000","last_clean":"2024-04-21T10:30:57.869277+0000","last_became_active":"2024-04-21T10:29:00.924016+0000","last_became_peered":"2024-04-21T10:29:00.924016+0000","last_unstale":"2024-04-21T10:30:57.869277+0000","last_undegraded":"2024-04-21T10:30:57.869277+0000","last_fullsized":"2024-04-21T10:30:57.869277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:49:24.004951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868884+0000","last_change":"2024-04-21T10:29:01.946315+0000","last_active":"2024-04-21T10:30:57.868884+0000","last_peered":"2024-04-21T10:30:57.868884+0000","last_clean":"2024-04-21T10:30:57.868884+0000","last_became_active":"2024-04-21T10:29:01.945920+0000","last_became_peered":"2024-04-21T10:29:01.945920+0000","last_unstale":"2024-04-21T10:30:57.868884+0000","last_undegraded":"2024-04-21T10:30:57.868884+0000","last_fullsized":"2024-04-21T10:30:57.868884+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:10:52.361615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807342+0000","last_change":"2024-04-21T10:29:00.919086+0000","last_active":"2024-04-21T10:30:57.807342+0000","last_peered":"2024-04-21T10:30:57.807342+0000","last_clean":"2024-04-21T10:30:57.807342+0000","last_became_active":"2024-04-21T10:29:00.918727+0000","last_became_peered":"2024-04-21T10:29:00.918727+0000","last_unstale":"2024-04-21T10:30:57.807342+0000","last_undegraded":"2024-04-21T10:30:57.807342+0000","last_fullsized":"2024-04-21T10:30:57.807342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:53:27.209135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869231+0000","last_change":"2024-04-21T10:29:01.944037+0000","last_active":"2024-04-21T10:30:57.869231+0000","last_peered":"2024-04-21T10:30:57.869231+0000","last_clean":"2024-04-21T10:30:57.869231+0000","last_became_active":"2024-04-21T10:29:01.943692+0000","last_became_peered":"2024-04-21T10:29:01.943692+0000","last_unstale":"2024-04-21T10:30:57.869231+0000","last_undegraded":"2024-04-21T10:30:57.869231+0000","last_fullsized":"2024-04-21T10:30:57.869231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:51.285535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868942+0000","last_change":"2024-04-21T10:29:00.930937+0000","last_active":"2024-04-21T10:30:57.868942+0000","last_peered":"2024-04-21T10:30:57.868942+0000","last_clean":"2024-04-21T10:30:57.868942+0000","last_became_active":"2024-04-21T10:29:00.930348+0000","last_became_peered":"2024-04-21T10:29:00.930348+0000","last_unstale":"2024-04-21T10:30:57.868942+0000","last_undegraded":"2024-04-21T10:30:57.868942+0000","last_fullsized":"2024-04-21T10:30:57.868942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:43:33.801432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868384+0000","last_change":"2024-04-21T10:29:01.937116+0000","last_active":"2024-04-21T10:30:57.868384+0000","last_peered":"2024-04-21T10:30:57.868384+0000","last_clean":"2024-04-21T10:30:57.868384+0000","last_became_active":"2024-04-21T10:29:01.936756+0000","last_became_peered":"2024-04-21T10:29:01.936756+0000","last_unstale":"2024-04-21T10:30:57.868384+0000","last_undegraded":"2024-04-21T10:30:57.868384+0000","last_fullsized":"2024-04-21T10:30:57.868384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:02.732399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868588+0000","last_change":"2024-04-21T10:30:03.312469+0000","last_active":"2024-04-21T10:30:57.868588+0000","last_peered":"2024-04-21T10:30:57.868588+0000","last_clean":"2024-04-21T10:30:57.868588+0000","last_became_active":"2024-04-21T10:30:03.310027+0000","last_became_peered":"2024-04-21T10:30:03.310027+0000","last_unstale":"2024-04-21T10:30:57.868588+0000","last_undegraded":"2024-04-21T10:30:57.868588+0000","last_fullsized":"2024-04-21T10:30:57.868588+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:12:18.582539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869264+0000","last_change":"2024-04-21T10:29:01.940677+0000","last_active":"2024-04-21T10:30:57.869264+0000","last_peered":"2024-04-21T10:30:57.869264+0000","last_clean":"2024-04-21T10:30:57.869264+0000","last_became_active":"2024-04-21T10:29:01.940296+0000","last_became_peered":"2024-04-21T10:29:01.940296+0000","last_unstale":"2024-04-21T10:30:57.869264+0000","last_undegraded":"2024-04-21T10:30:57.869264+0000","last_fullsized":"2024-04-21T10:30:57.869264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:24:52.662041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868754+0000","last_change":"2024-04-21T10:29:00.924392+0000","last_active":"2024-04-21T10:30:57.868754+0000","last_peered":"2024-04-21T10:30:57.868754+0000","last_clean":"2024-04-21T10:30:57.868754+0000","last_became_active":"2024-04-21T10:29:00.924029+0000","last_became_peered":"2024-04-21T10:29:00.924029+0000","last_unstale":"2024-04-21T10:30:57.868754+0000","last_undegraded":"2024-04-21T10:30:57.868754+0000","last_fullsized":"2024-04-21T10:30:57.868754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:23:50.726978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869200+0000","last_change":"2024-04-21T10:29:01.939687+0000","last_active":"2024-04-21T10:30:57.869200+0000","last_peered":"2024-04-21T10:30:57.869200+0000","last_clean":"2024-04-21T10:30:57.869200+0000","last_became_active":"2024-04-21T10:29:01.939343+0000","last_became_peered":"2024-04-21T10:29:01.939343+0000","last_unstale":"2024-04-21T10:30:57.869200+0000","last_undegraded":"2024-04-21T10:30:57.869200+0000","last_fullsized":"2024-04-21T10:30:57.869200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:50.481998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869578+0000","last_change":"2024-04-21T10:29:00.923115+0000","last_active":"2024-04-21T10:30:57.869578+0000","last_peered":"2024-04-21T10:30:57.869578+0000","last_clean":"2024-04-21T10:30:57.869578+0000","last_became_active":"2024-04-21T10:29:00.922707+0000","last_became_peered":"2024-04-21T10:29:00.922707+0000","last_unstale":"2024-04-21T10:30:57.869578+0000","last_undegraded":"2024-04-21T10:30:57.869578+0000","last_fullsized":"2024-04-21T10:30:57.869578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:55:53.419904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869663+0000","last_change":"2024-04-21T10:29:01.940666+0000","last_active":"2024-04-21T10:30:57.869663+0000","last_peered":"2024-04-21T10:30:57.869663+0000","last_clean":"2024-04-21T10:30:57.869663+0000","last_became_active":"2024-04-21T10:29:01.940331+0000","last_became_peered":"2024-04-21T10:29:01.940331+0000","last_unstale":"2024-04-21T10:30:57.869663+0000","last_undegraded":"2024-04-21T10:30:57.869663+0000","last_fullsized":"2024-04-21T10:30:57.869663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:52:41.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'39","reported_seq":68,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.006266+0000","last_change":"2024-04-21T10:29:00.927873+0000","last_active":"2024-04-21T10:30:58.006266+0000","last_peered":"2024-04-21T10:30:58.006266+0000","last_clean":"2024-04-21T10:30:58.006266+0000","last_became_active":"2024-04-21T10:29:00.927403+0000","last_became_peered":"2024-04-21T10:29:00.927403+0000","last_unstale":"2024-04-21T10:30:58.006266+0000","last_undegraded":"2024-04-21T10:30:58.006266+0000","last_fullsized":"2024-04-21T10:30:58.006266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:05:05.276978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869023+0000","last_change":"2024-04-21T10:29:01.944725+0000","last_active":"2024-04-21T10:30:57.869023+0000","last_peered":"2024-04-21T10:30:57.869023+0000","last_clean":"2024-04-21T10:30:57.869023+0000","last_became_active":"2024-04-21T10:29:01.944326+0000","last_became_peered":"2024-04-21T10:29:01.944326+0000","last_unstale":"2024-04-21T10:30:57.869023+0000","last_undegraded":"2024-04-21T10:30:57.869023+0000","last_fullsized":"2024-04-21T10:30:57.869023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:50.209064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.806132+0000","last_change":"2024-04-21T10:29:00.924694+0000","last_active":"2024-04-21T10:30:57.806132+0000","last_peered":"2024-04-21T10:30:57.806132+0000","last_clean":"2024-04-21T10:30:57.806132+0000","last_became_active":"2024-04-21T10:29:00.924373+0000","last_became_peered":"2024-04-21T10:29:00.924373+0000","last_unstale":"2024-04-21T10:30:57.806132+0000","last_undegraded":"2024-04-21T10:30:57.806132+0000","last_fullsized":"2024-04-21T10:30:57.806132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:10:14.681630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869611+0000","last_change":"2024-04-21T10:29:01.939277+0000","last_active":"2024-04-21T10:30:57.869611+0000","last_peered":"2024-04-21T10:30:57.869611+0000","last_clean":"2024-04-21T10:30:57.869611+0000","last_became_active":"2024-04-21T10:29:01.938844+0000","last_became_peered":"2024-04-21T10:29:01.938844+0000","last_unstale":"2024-04-21T10:30:57.869611+0000","last_undegraded":"2024-04-21T10:30:57.869611+0000","last_fullsized":"2024-04-21T10:30:57.869611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:38:14.553868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868465+0000","last_change":"2024-04-21T10:29:01.942654+0000","last_active":"2024-04-21T10:30:57.868465+0000","last_peered":"2024-04-21T10:30:57.868465+0000","last_clean":"2024-04-21T10:30:57.868465+0000","last_became_active":"2024-04-21T10:29:01.942324+0000","last_became_peered":"2024-04-21T10:29:01.942324+0000","last_unstale":"2024-04-21T10:30:57.868465+0000","last_undegraded":"2024-04-21T10:30:57.868465+0000","last_fullsized":"2024-04-21T10:30:57.868465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:19.027138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868724+0000","last_change":"2024-04-21T10:30:03.307328+0000","last_active":"2024-04-21T10:30:57.868724+0000","last_peered":"2024-04-21T10:30:57.868724+0000","last_clean":"2024-04-21T10:30:57.868724+0000","last_became_active":"2024-04-21T10:30:03.306976+0000","last_became_peered":"2024-04-21T10:30:03.306976+0000","last_unstale":"2024-04-21T10:30:57.868724+0000","last_undegraded":"2024-04-21T10:30:57.868724+0000","last_fullsized":"2024-04-21T10:30:57.868724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:19:31.739372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868933+0000","last_change":"2024-04-21T10:29:01.944405+0000","last_active":"2024-04-21T10:30:57.868933+0000","last_peered":"2024-04-21T10:30:57.868933+0000","last_clean":"2024-04-21T10:30:57.868933+0000","last_became_active":"2024-04-21T10:29:01.944085+0000","last_became_peered":"2024-04-21T10:29:01.944085+0000","last_unstale":"2024-04-21T10:30:57.868933+0000","last_undegraded":"2024-04-21T10:30:57.868933+0000","last_fullsized":"2024-04-21T10:30:57.868933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:13.299859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868270+0000","last_change":"2024-04-21T10:29:01.941054+0000","last_active":"2024-04-21T10:30:57.868270+0000","last_peered":"2024-04-21T10:30:57.868270+0000","last_clean":"2024-04-21T10:30:57.868270+0000","last_became_active":"2024-04-21T10:29:01.940243+0000","last_became_peered":"2024-04-21T10:29:01.940243+0000","last_unstale":"2024-04-21T10:30:57.868270+0000","last_undegraded":"2024-04-21T10:30:57.868270+0000","last_fullsized":"2024-04-21T10:30:57.868270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:31:04.222026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805890+0000","last_change":"2024-04-21T10:30:03.307984+0000","last_active":"2024-04-21T10:30:57.805890+0000","last_peered":"2024-04-21T10:30:57.805890+0000","last_clean":"2024-04-21T10:30:57.805890+0000","last_became_active":"2024-04-21T10:30:03.307584+0000","last_became_peered":"2024-04-21T10:30:03.307584+0000","last_unstale":"2024-04-21T10:30:57.805890+0000","last_undegraded":"2024-04-21T10:30:57.805890+0000","last_fullsized":"2024-04-21T10:30:57.805890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:12.699665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868858+0000","last_change":"2024-04-21T10:29:01.947268+0000","last_active":"2024-04-21T10:30:57.868858+0000","last_peered":"2024-04-21T10:30:57.868858+0000","last_clean":"2024-04-21T10:30:57.868858+0000","last_became_active":"2024-04-21T10:29:01.946921+0000","last_became_peered":"2024-04-21T10:29:01.946921+0000","last_unstale":"2024-04-21T10:30:57.868858+0000","last_undegraded":"2024-04-21T10:30:57.868858+0000","last_fullsized":"2024-04-21T10:30:57.868858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:50:43.784920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869679+0000","last_change":"2024-04-21T10:29:01.949969+0000","last_active":"2024-04-21T10:30:57.869679+0000","last_peered":"2024-04-21T10:30:57.869679+0000","last_clean":"2024-04-21T10:30:57.869679+0000","last_became_active":"2024-04-21T10:29:01.949487+0000","last_became_peered":"2024-04-21T10:29:01.949487+0000","last_unstale":"2024-04-21T10:30:57.869679+0000","last_undegraded":"2024-04-21T10:30:57.869679+0000","last_fullsized":"2024-04-21T10:30:57.869679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:26:59.620209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869496+0000","last_change":"2024-04-21T10:29:01.937904+0000","last_active":"2024-04-21T10:30:57.869496+0000","last_peered":"2024-04-21T10:30:57.869496+0000","last_clean":"2024-04-21T10:30:57.869496+0000","last_became_active":"2024-04-21T10:29:01.937504+0000","last_became_peered":"2024-04-21T10:29:01.937504+0000","last_unstale":"2024-04-21T10:30:57.869496+0000","last_undegraded":"2024-04-21T10:30:57.869496+0000","last_fullsized":"2024-04-21T10:30:57.869496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:58:24.245118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807511+0000","last_change":"2024-04-21T10:30:03.310042+0000","last_active":"2024-04-21T10:30:57.807511+0000","last_peered":"2024-04-21T10:30:57.807511+0000","last_clean":"2024-04-21T10:30:57.807511+0000","last_became_active":"2024-04-21T10:30:03.309663+0000","last_became_peered":"2024-04-21T10:30:03.309663+0000","last_unstale":"2024-04-21T10:30:57.807511+0000","last_undegraded":"2024-04-21T10:30:57.807511+0000","last_fullsized":"2024-04-21T10:30:57.807511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:19.906949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868797+0000","last_change":"2024-04-21T10:29:01.936880+0000","last_active":"2024-04-21T10:30:57.868797+0000","last_peered":"2024-04-21T10:30:57.868797+0000","last_clean":"2024-04-21T10:30:57.868797+0000","last_became_active":"2024-04-21T10:29:01.936555+0000","last_became_peered":"2024-04-21T10:29:01.936555+0000","last_unstale":"2024-04-21T10:30:57.868797+0000","last_undegraded":"2024-04-21T10:30:57.868797+0000","last_fullsized":"2024-04-21T10:30:57.868797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:32.831606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807441+0000","last_change":"2024-04-21T10:29:01.943238+0000","last_active":"2024-04-21T10:30:57.807441+0000","last_peered":"2024-04-21T10:30:57.807441+0000","last_clean":"2024-04-21T10:30:57.807441+0000","last_became_active":"2024-04-21T10:29:01.942841+0000","last_became_peered":"2024-04-21T10:29:01.942841+0000","last_unstale":"2024-04-21T10:30:57.807441+0000","last_undegraded":"2024-04-21T10:30:57.807441+0000","last_fullsized":"2024-04-21T10:30:57.807441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:59:48.551524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805955+0000","last_change":"2024-04-21T10:29:01.933712+0000","last_active":"2024-04-21T10:30:57.805955+0000","last_peered":"2024-04-21T10:30:57.805955+0000","last_clean":"2024-04-21T10:30:57.805955+0000","last_became_active":"2024-04-21T10:29:01.933399+0000","last_became_peered":"2024-04-21T10:29:01.933399+0000","last_unstale":"2024-04-21T10:30:57.805955+0000","last_undegraded":"2024-04-21T10:30:57.805955+0000","last_fullsized":"2024-04-21T10:30:57.805955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:34:16.346403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868641+0000","last_change":"2024-04-21T10:29:01.947225+0000","last_active":"2024-04-21T10:30:57.868641+0000","last_peered":"2024-04-21T10:30:57.868641+0000","last_clean":"2024-04-21T10:30:57.868641+0000","last_became_active":"2024-04-21T10:29:01.946890+0000","last_became_peered":"2024-04-21T10:29:01.946890+0000","last_unstale":"2024-04-21T10:30:57.868641+0000","last_undegraded":"2024-04-21T10:30:57.868641+0000","last_fullsized":"2024-04-21T10:30:57.868641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:19:00.960767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597257,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":801,"num_read_kb":508,"num_write":286,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1791771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318309,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":472945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579945,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28800,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716352,"statfs":{"total":95995035648,"available":95965544448,"internally_reserved":0,"allocated":1826816,"data_stored":1263963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743470,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28384,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716768,"statfs":{"total":95995035648,"available":95965970432,"internally_reserved":0,"allocated":1396736,"data_stored":866285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005107,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28376,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716776,"statfs":{"total":95995035648,"available":95965978624,"internally_reserved":0,"allocated":1388544,"data_stored":866093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1066624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430269,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":472458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":539098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":473729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":541006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1536000,"data_stored":997585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":995328,"data_stored":473427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1609728,"data_stored":1064212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":393842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T10:31:03.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T10:31:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: Updating smithi026:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:31:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:31:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:31:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:31:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: Updating smithi026:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/4103551741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:31:03.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:31:03.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:31:03.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3323333739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:31:03.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3094969291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:31:03.700 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:03 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3453980540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:31:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.735+0000 7f0ad890e640 1 -- 172.21.15.26:0/2016166838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad41017f0 msgr2=0x7f0ad410dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.735+0000 7f0ad890e640 1 --2- 172.21.15.26:0/2016166838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad41017f0 0x7f0ad410dcb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009ca0 tx=0x7f0acc02c180 comp rx=0 tx=0).stop 2024-04-21T10:31:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.735+0000 7f0ad890e640 1 -- 172.21.15.26:0/2016166838 shutdown_connections 2024-04-21T10:31:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.735+0000 7f0ad890e640 1 --2- 172.21.15.26:0/2016166838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad41017f0 0x7f0ad410dcb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.735+0000 7f0ad890e640 1 --2- 172.21.15.26:0/2016166838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ad4100e50 0x7f0ad41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.735+0000 7f0ad890e640 1 --2- 172.21.15.26:0/2016166838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ad4106f10 0x7f0ad41072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.735+0000 7f0ad890e640 1 -- 172.21.15.26:0/2016166838 >> 172.21.15.26:0/2016166838 conn(0x7f0ad4077f80 msgr2=0x7f0ad4078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.735+0000 7f0ad890e640 1 -- 172.21.15.26:0/2016166838 shutdown_connections 2024-04-21T10:31:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.735+0000 7f0ad890e640 1 -- 172.21.15.26:0/2016166838 wait complete. 2024-04-21T10:31:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad890e640 1 Processor -- start 2024-04-21T10:31:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad890e640 1 -- start start 2024-04-21T10:31:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad890e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad4100e50 0x7f0ad4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad890e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ad41017f0 0x7f0ad4198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad890e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ad4106f10 0x7f0ad419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad890e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad41110a0 con 0x7f0ad4100e50 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad890e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad4110f20 con 0x7f0ad4106f10 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad890e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad4111220 con 0x7f0ad41017f0 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad4100e50 0x7f0ad4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad4100e50 0x7f0ad4198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46432/0 (socket says 172.21.15.26:46432) 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad259c640 1 -- 172.21.15.26:0/3634185359 learned_addr learned my addr 172.21.15.26:0/3634185359 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0acbfff640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ad41017f0 0x7f0ad4198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad259c640 1 -- 172.21.15.26:0/3634185359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ad41017f0 msgr2=0x7f0ad4198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad259c640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ad41017f0 0x7f0ad4198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad2d9d640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ad4106f10 0x7f0ad419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad259c640 1 -- 172.21.15.26:0/3634185359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ad4106f10 msgr2=0x7f0ad419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad259c640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ad4106f10 0x7f0ad419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad259c640 1 -- 172.21.15.26:0/3634185359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad419d640 con 0x7f0ad4100e50 2024-04-21T10:31:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.736+0000 7f0ad259c640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad4100e50 0x7f0ad4198650 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400c970 tx=0x7f0ac400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.737+0000 7f0acb7fe640 1 -- 172.21.15.26:0/3634185359 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4007bb0 con 0x7f0ad4100e50 2024-04-21T10:31:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.737+0000 7f0ad890e640 1 -- 172.21.15.26:0/3634185359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad419d930 con 0x7f0ad4100e50 2024-04-21T10:31:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.737+0000 7f0acb7fe640 1 -- 172.21.15.26:0/3634185359 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ac4007d50 con 0x7f0ad4100e50 2024-04-21T10:31:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.737+0000 7f0acb7fe640 1 -- 172.21.15.26:0/3634185359 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac400f450 con 0x7f0ad4100e50 2024-04-21T10:31:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.737+0000 7f0ad890e640 1 -- 172.21.15.26:0/3634185359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad41a51c0 con 0x7f0ad4100e50 2024-04-21T10:31:03.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.738+0000 7f0ad890e640 1 -- 172.21.15.26:0/3634185359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad4068550 con 0x7f0ad4100e50 2024-04-21T10:31:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.739+0000 7f0acb7fe640 1 -- 172.21.15.26:0/3634185359 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f0ac4004f20 con 0x7f0ad4100e50 2024-04-21T10:31:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.739+0000 7f0acb7fe640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0ac00a3ed0 0x7f0ac00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.740+0000 7f0acbfff640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0ac00a3ed0 0x7f0ac00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.740+0000 7f0acb7fe640 1 -- 172.21.15.26:0/3634185359 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f0ac40d0980 con 0x7f0ad4100e50 2024-04-21T10:31:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.740+0000 7f0acbfff640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0ac00a3ed0 0x7f0ac00a6390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0abc005fd0 tx=0x7f0abc005c70 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.744+0000 7f0acb7fe640 1 -- 172.21.15.26:0/3634185359 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ac4014030 con 0x7f0ad4100e50 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: Updating smithi026:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: Updating smithi026:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: Updating smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: Updating smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/4103551741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-21T10:31:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3323333739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3094969291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:31:03.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:03 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3453980540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:31:03.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.946+0000 7f0ad890e640 1 -- 172.21.15.26:0/3634185359 --> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ad406a2a0 con 0x7f0ac00a3ed0 2024-04-21T10:31:03.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.954+0000 7f0acb7fe640 1 -- 172.21.15.26:0/3634185359 <== mgr.24869 v2:172.21.15.26:6800/4245046040 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257738 (secure 0 0 0) 0x7f0ad406a2a0 con 0x7f0ac00a3ed0 2024-04-21T10:31:03.957 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-21T10:31:03.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.958+0000 7f0ad890e640 1 -- 172.21.15.26:0/3634185359 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0ac00a3ed0 msgr2=0x7f0ac00a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.958+0000 7f0ad890e640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0ac00a3ed0 0x7f0ac00a6390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0abc005fd0 tx=0x7f0abc005c70 comp rx=0 tx=0).stop 2024-04-21T10:31:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.958+0000 7f0ad890e640 1 -- 172.21.15.26:0/3634185359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad4100e50 msgr2=0x7f0ad4198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.958+0000 7f0ad890e640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad4100e50 0x7f0ad4198650 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400c970 tx=0x7f0ac400ce30 comp rx=0 tx=0).stop 2024-04-21T10:31:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.958+0000 7f0ad890e640 1 -- 172.21.15.26:0/3634185359 shutdown_connections 2024-04-21T10:31:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.958+0000 7f0ad890e640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0ac00a3ed0 0x7f0ac00a6390 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.958+0000 7f0ad890e640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ad4106f10 0x7f0ad419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.958+0000 7f0ad890e640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ad41017f0 0x7f0ad4198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.958+0000 7f0ad890e640 1 --2- 172.21.15.26:0/3634185359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ad4100e50 0x7f0ad4198650 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.958+0000 7f0ad890e640 1 -- 172.21.15.26:0/3634185359 >> 172.21.15.26:0/3634185359 conn(0x7f0ad4077f80 msgr2=0x7f0ad41000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.958+0000 7f0ad890e640 1 -- 172.21.15.26:0/3634185359 shutdown_connections 2024-04-21T10:31:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:03.958+0000 7f0ad890e640 1 -- 172.21.15.26:0/3634185359 wait complete. 2024-04-21T10:31:03.969 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-21T10:31:03.820090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187859,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":942,"num_read_kb":632,"num_write":469,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364572,"kb_used_data":15324,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124577252,"statfs":{"total":1151940427776,"available":1151567106048,"internally_reserved":0,"allocated":15691776,"data_stored":9294057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.989910"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869187+0000","last_change":"2024-04-21T10:29:01.940996+0000","last_active":"2024-04-21T10:30:57.869187+0000","last_peered":"2024-04-21T10:30:57.869187+0000","last_clean":"2024-04-21T10:30:57.869187+0000","last_became_active":"2024-04-21T10:29:01.940662+0000","last_became_peered":"2024-04-21T10:29:01.940662+0000","last_unstale":"2024-04-21T10:30:57.869187+0000","last_undegraded":"2024-04-21T10:30:57.869187+0000","last_fullsized":"2024-04-21T10:30:57.869187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:16:01.434895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868745+0000","last_change":"2024-04-21T10:29:01.939238+0000","last_active":"2024-04-21T10:30:57.868745+0000","last_peered":"2024-04-21T10:30:57.868745+0000","last_clean":"2024-04-21T10:30:57.868745+0000","last_became_active":"2024-04-21T10:29:01.938907+0000","last_became_peered":"2024-04-21T10:29:01.938907+0000","last_unstale":"2024-04-21T10:30:57.868745+0000","last_undegraded":"2024-04-21T10:30:57.868745+0000","last_fullsized":"2024-04-21T10:30:57.868745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:09.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869242+0000","last_change":"2024-04-21T10:29:01.932338+0000","last_active":"2024-04-21T10:30:57.869242+0000","last_peered":"2024-04-21T10:30:57.869242+0000","last_clean":"2024-04-21T10:30:57.869242+0000","last_became_active":"2024-04-21T10:29:01.931916+0000","last_became_peered":"2024-04-21T10:29:01.931916+0000","last_unstale":"2024-04-21T10:30:57.869242+0000","last_undegraded":"2024-04-21T10:30:57.869242+0000","last_fullsized":"2024-04-21T10:30:57.869242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:52.859695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869822+0000","last_change":"2024-04-21T10:29:01.945572+0000","last_active":"2024-04-21T10:30:57.869822+0000","last_peered":"2024-04-21T10:30:57.869822+0000","last_clean":"2024-04-21T10:30:57.869822+0000","last_became_active":"2024-04-21T10:29:01.940114+0000","last_became_peered":"2024-04-21T10:29:01.940114+0000","last_unstale":"2024-04-21T10:30:57.869822+0000","last_undegraded":"2024-04-21T10:30:57.869822+0000","last_fullsized":"2024-04-21T10:30:57.869822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:21.935573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868608+0000","last_change":"2024-04-21T10:29:01.946021+0000","last_active":"2024-04-21T10:30:57.868608+0000","last_peered":"2024-04-21T10:30:57.868608+0000","last_clean":"2024-04-21T10:30:57.868608+0000","last_became_active":"2024-04-21T10:29:01.945637+0000","last_became_peered":"2024-04-21T10:29:01.945637+0000","last_unstale":"2024-04-21T10:30:57.868608+0000","last_undegraded":"2024-04-21T10:30:57.868608+0000","last_fullsized":"2024-04-21T10:30:57.868608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:20.840494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868714+0000","last_change":"2024-04-21T10:29:01.935402+0000","last_active":"2024-04-21T10:30:57.868714+0000","last_peered":"2024-04-21T10:30:57.868714+0000","last_clean":"2024-04-21T10:30:57.868714+0000","last_became_active":"2024-04-21T10:29:01.934994+0000","last_became_peered":"2024-04-21T10:29:01.934994+0000","last_unstale":"2024-04-21T10:30:57.868714+0000","last_undegraded":"2024-04-21T10:30:57.868714+0000","last_fullsized":"2024-04-21T10:30:57.868714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:18.320530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868977+0000","last_change":"2024-04-21T10:29:01.943185+0000","last_active":"2024-04-21T10:30:57.868977+0000","last_peered":"2024-04-21T10:30:57.868977+0000","last_clean":"2024-04-21T10:30:57.868977+0000","last_became_active":"2024-04-21T10:29:01.942805+0000","last_became_peered":"2024-04-21T10:29:01.942805+0000","last_unstale":"2024-04-21T10:30:57.868977+0000","last_undegraded":"2024-04-21T10:30:57.868977+0000","last_fullsized":"2024-04-21T10:30:57.868977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:06.776472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868710+0000","last_change":"2024-04-21T10:29:01.940995+0000","last_active":"2024-04-21T10:30:57.868710+0000","last_peered":"2024-04-21T10:30:57.868710+0000","last_clean":"2024-04-21T10:30:57.868710+0000","last_became_active":"2024-04-21T10:29:01.939173+0000","last_became_peered":"2024-04-21T10:29:01.939173+0000","last_unstale":"2024-04-21T10:30:57.868710+0000","last_undegraded":"2024-04-21T10:30:57.868710+0000","last_fullsized":"2024-04-21T10:30:57.868710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.740147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807868+0000","last_change":"2024-04-21T10:29:01.944284+0000","last_active":"2024-04-21T10:30:57.807868+0000","last_peered":"2024-04-21T10:30:57.807868+0000","last_clean":"2024-04-21T10:30:57.807868+0000","last_became_active":"2024-04-21T10:29:01.943965+0000","last_became_peered":"2024-04-21T10:29:01.943965+0000","last_unstale":"2024-04-21T10:30:57.807868+0000","last_undegraded":"2024-04-21T10:30:57.807868+0000","last_fullsized":"2024-04-21T10:30:57.807868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:08:54.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869273+0000","last_change":"2024-04-21T10:29:01.936250+0000","last_active":"2024-04-21T10:30:57.869273+0000","last_peered":"2024-04-21T10:30:57.869273+0000","last_clean":"2024-04-21T10:30:57.869273+0000","last_became_active":"2024-04-21T10:29:01.935904+0000","last_became_peered":"2024-04-21T10:29:01.935904+0000","last_unstale":"2024-04-21T10:30:57.869273+0000","last_undegraded":"2024-04-21T10:30:57.869273+0000","last_fullsized":"2024-04-21T10:30:57.869273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:35.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869510+0000","last_change":"2024-04-21T10:29:01.946550+0000","last_active":"2024-04-21T10:30:57.869510+0000","last_peered":"2024-04-21T10:30:57.869510+0000","last_clean":"2024-04-21T10:30:57.869510+0000","last_became_active":"2024-04-21T10:29:01.946170+0000","last_became_peered":"2024-04-21T10:29:01.946170+0000","last_unstale":"2024-04-21T10:30:57.869510+0000","last_undegraded":"2024-04-21T10:30:57.869510+0000","last_fullsized":"2024-04-21T10:30:57.869510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:25:47.671821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869011+0000","last_change":"2024-04-21T10:29:01.946036+0000","last_active":"2024-04-21T10:30:57.869011+0000","last_peered":"2024-04-21T10:30:57.869011+0000","last_clean":"2024-04-21T10:30:57.869011+0000","last_became_active":"2024-04-21T10:29:01.940705+0000","last_became_peered":"2024-04-21T10:29:01.940705+0000","last_unstale":"2024-04-21T10:30:57.869011+0000","last_undegraded":"2024-04-21T10:30:57.869011+0000","last_fullsized":"2024-04-21T10:30:57.869011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:11:33.196059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805947+0000","last_change":"2024-04-21T10:29:01.940438+0000","last_active":"2024-04-21T10:30:57.805947+0000","last_peered":"2024-04-21T10:30:57.805947+0000","last_clean":"2024-04-21T10:30:57.805947+0000","last_became_active":"2024-04-21T10:29:01.940085+0000","last_became_peered":"2024-04-21T10:29:01.940085+0000","last_unstale":"2024-04-21T10:30:57.805947+0000","last_undegraded":"2024-04-21T10:30:57.805947+0000","last_fullsized":"2024-04-21T10:30:57.805947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:22:47.828858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868102+0000","last_change":"2024-04-21T10:29:01.941667+0000","last_active":"2024-04-21T10:30:57.868102+0000","last_peered":"2024-04-21T10:30:57.868102+0000","last_clean":"2024-04-21T10:30:57.868102+0000","last_became_active":"2024-04-21T10:29:01.940745+0000","last_became_peered":"2024-04-21T10:29:01.940745+0000","last_unstale":"2024-04-21T10:30:57.868102+0000","last_undegraded":"2024-04-21T10:30:57.868102+0000","last_fullsized":"2024-04-21T10:30:57.868102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:45:51.825780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.867997+0000","last_change":"2024-04-21T10:29:01.947771+0000","last_active":"2024-04-21T10:30:57.867997+0000","last_peered":"2024-04-21T10:30:57.867997+0000","last_clean":"2024-04-21T10:30:57.867997+0000","last_became_active":"2024-04-21T10:29:01.946818+0000","last_became_peered":"2024-04-21T10:29:01.946818+0000","last_unstale":"2024-04-21T10:30:57.867997+0000","last_undegraded":"2024-04-21T10:30:57.867997+0000","last_fullsized":"2024-04-21T10:30:57.867997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:18:42.159152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807344+0000","last_change":"2024-04-21T10:29:01.942499+0000","last_active":"2024-04-21T10:30:57.807344+0000","last_peered":"2024-04-21T10:30:57.807344+0000","last_clean":"2024-04-21T10:30:57.807344+0000","last_became_active":"2024-04-21T10:29:01.942136+0000","last_became_peered":"2024-04-21T10:29:01.942136+0000","last_unstale":"2024-04-21T10:30:57.807344+0000","last_undegraded":"2024-04-21T10:30:57.807344+0000","last_fullsized":"2024-04-21T10:30:57.807344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:34:11.822230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869190+0000","last_change":"2024-04-21T10:29:01.948822+0000","last_active":"2024-04-21T10:30:57.869190+0000","last_peered":"2024-04-21T10:30:57.869190+0000","last_clean":"2024-04-21T10:30:57.869190+0000","last_became_active":"2024-04-21T10:29:01.948492+0000","last_became_peered":"2024-04-21T10:29:01.948492+0000","last_unstale":"2024-04-21T10:30:57.869190+0000","last_undegraded":"2024-04-21T10:30:57.869190+0000","last_fullsized":"2024-04-21T10:30:57.869190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:06.992585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869064+0000","last_change":"2024-04-21T10:29:01.931265+0000","last_active":"2024-04-21T10:30:57.869064+0000","last_peered":"2024-04-21T10:30:57.869064+0000","last_clean":"2024-04-21T10:30:57.869064+0000","last_became_active":"2024-04-21T10:29:01.930877+0000","last_became_peered":"2024-04-21T10:29:01.930877+0000","last_unstale":"2024-04-21T10:30:57.869064+0000","last_undegraded":"2024-04-21T10:30:57.869064+0000","last_fullsized":"2024-04-21T10:30:57.869064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:49:00.951226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869590+0000","last_change":"2024-04-21T10:29:01.941718+0000","last_active":"2024-04-21T10:30:57.869590+0000","last_peered":"2024-04-21T10:30:57.869590+0000","last_clean":"2024-04-21T10:30:57.869590+0000","last_became_active":"2024-04-21T10:29:01.941386+0000","last_became_peered":"2024-04-21T10:29:01.941386+0000","last_unstale":"2024-04-21T10:30:57.869590+0000","last_undegraded":"2024-04-21T10:30:57.869590+0000","last_fullsized":"2024-04-21T10:30:57.869590+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:51:29.847961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869350+0000","last_change":"2024-04-21T10:29:00.923670+0000","last_active":"2024-04-21T10:30:57.869350+0000","last_peered":"2024-04-21T10:30:57.869350+0000","last_clean":"2024-04-21T10:30:57.869350+0000","last_became_active":"2024-04-21T10:29:00.923100+0000","last_became_peered":"2024-04-21T10:29:00.923100+0000","last_unstale":"2024-04-21T10:30:57.869350+0000","last_undegraded":"2024-04-21T10:30:57.869350+0000","last_fullsized":"2024-04-21T10:30:57.869350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:45.692961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869435+0000","last_change":"2024-04-21T10:29:01.938131+0000","last_active":"2024-04-21T10:30:57.869435+0000","last_peered":"2024-04-21T10:30:57.869435+0000","last_clean":"2024-04-21T10:30:57.869435+0000","last_became_active":"2024-04-21T10:29:01.937752+0000","last_became_peered":"2024-04-21T10:29:01.937752+0000","last_unstale":"2024-04-21T10:30:57.869435+0000","last_undegraded":"2024-04-21T10:30:57.869435+0000","last_fullsized":"2024-04-21T10:30:57.869435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:06:49.383554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869517+0000","last_change":"2024-04-21T10:29:00.929877+0000","last_active":"2024-04-21T10:30:57.869517+0000","last_peered":"2024-04-21T10:30:57.869517+0000","last_clean":"2024-04-21T10:30:57.869517+0000","last_became_active":"2024-04-21T10:29:00.929564+0000","last_became_peered":"2024-04-21T10:29:00.929564+0000","last_unstale":"2024-04-21T10:30:57.869517+0000","last_undegraded":"2024-04-21T10:30:57.869517+0000","last_fullsized":"2024-04-21T10:30:57.869517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:35.617533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869262+0000","last_change":"2024-04-21T10:29:01.942782+0000","last_active":"2024-04-21T10:30:57.869262+0000","last_peered":"2024-04-21T10:30:57.869262+0000","last_clean":"2024-04-21T10:30:57.869262+0000","last_became_active":"2024-04-21T10:29:01.940501+0000","last_became_peered":"2024-04-21T10:29:01.940501+0000","last_unstale":"2024-04-21T10:30:57.869262+0000","last_undegraded":"2024-04-21T10:30:57.869262+0000","last_fullsized":"2024-04-21T10:30:57.869262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:04:03.431016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869465+0000","last_change":"2024-04-21T10:29:00.924395+0000","last_active":"2024-04-21T10:30:57.869465+0000","last_peered":"2024-04-21T10:30:57.869465+0000","last_clean":"2024-04-21T10:30:57.869465+0000","last_became_active":"2024-04-21T10:29:00.924045+0000","last_became_peered":"2024-04-21T10:29:00.924045+0000","last_unstale":"2024-04-21T10:30:57.869465+0000","last_undegraded":"2024-04-21T10:30:57.869465+0000","last_fullsized":"2024-04-21T10:30:57.869465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:08:53.922213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807373+0000","last_change":"2024-04-21T10:29:01.934508+0000","last_active":"2024-04-21T10:30:57.807373+0000","last_peered":"2024-04-21T10:30:57.807373+0000","last_clean":"2024-04-21T10:30:57.807373+0000","last_became_active":"2024-04-21T10:29:01.934051+0000","last_became_peered":"2024-04-21T10:29:01.934051+0000","last_unstale":"2024-04-21T10:30:57.807373+0000","last_undegraded":"2024-04-21T10:30:57.807373+0000","last_fullsized":"2024-04-21T10:30:57.807373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:49:27.686603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868924+0000","last_change":"2024-04-21T10:29:00.925850+0000","last_active":"2024-04-21T10:30:57.868924+0000","last_peered":"2024-04-21T10:30:57.868924+0000","last_clean":"2024-04-21T10:30:57.868924+0000","last_became_active":"2024-04-21T10:29:00.925521+0000","last_became_peered":"2024-04-21T10:29:00.925521+0000","last_unstale":"2024-04-21T10:30:57.868924+0000","last_undegraded":"2024-04-21T10:30:57.868924+0000","last_fullsized":"2024-04-21T10:30:57.868924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:36:55.403413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807733+0000","last_change":"2024-04-21T10:29:01.930046+0000","last_active":"2024-04-21T10:30:57.807733+0000","last_peered":"2024-04-21T10:30:57.807733+0000","last_clean":"2024-04-21T10:30:57.807733+0000","last_became_active":"2024-04-21T10:29:01.929736+0000","last_became_peered":"2024-04-21T10:29:01.929736+0000","last_unstale":"2024-04-21T10:30:57.807733+0000","last_undegraded":"2024-04-21T10:30:57.807733+0000","last_fullsized":"2024-04-21T10:30:57.807733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:11:10.833267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":143,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.047873+0000","last_change":"2024-04-21T10:29:00.927936+0000","last_active":"2024-04-21T10:30:58.047873+0000","last_peered":"2024-04-21T10:30:58.047873+0000","last_clean":"2024-04-21T10:30:58.047873+0000","last_became_active":"2024-04-21T10:29:00.927352+0000","last_became_peered":"2024-04-21T10:29:00.927352+0000","last_unstale":"2024-04-21T10:30:58.047873+0000","last_undegraded":"2024-04-21T10:30:58.047873+0000","last_fullsized":"2024-04-21T10:30:58.047873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:14:10.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869207+0000","last_change":"2024-04-21T10:30:03.306538+0000","last_active":"2024-04-21T10:30:57.869207+0000","last_peered":"2024-04-21T10:30:57.869207+0000","last_clean":"2024-04-21T10:30:57.869207+0000","last_became_active":"2024-04-21T10:30:03.306197+0000","last_became_peered":"2024-04-21T10:30:03.306197+0000","last_unstale":"2024-04-21T10:30:57.869207+0000","last_undegraded":"2024-04-21T10:30:57.869207+0000","last_fullsized":"2024-04-21T10:30:57.869207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:28:25.688821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868668+0000","last_change":"2024-04-21T10:30:03.522106+0000","last_active":"2024-04-21T10:30:57.868668+0000","last_peered":"2024-04-21T10:30:57.868668+0000","last_clean":"2024-04-21T10:30:57.868668+0000","last_became_active":"2024-04-21T10:30:03.519189+0000","last_became_peered":"2024-04-21T10:30:03.519189+0000","last_unstale":"2024-04-21T10:30:57.868668+0000","last_undegraded":"2024-04-21T10:30:57.868668+0000","last_fullsized":"2024-04-21T10:30:57.868668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:49:39.930818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.808006+0000","last_change":"2024-04-21T10:29:01.918627+0000","last_active":"2024-04-21T10:30:57.808006+0000","last_peered":"2024-04-21T10:30:57.808006+0000","last_clean":"2024-04-21T10:30:57.808006+0000","last_became_active":"2024-04-21T10:29:01.917993+0000","last_became_peered":"2024-04-21T10:29:01.917993+0000","last_unstale":"2024-04-21T10:30:57.808006+0000","last_undegraded":"2024-04-21T10:30:57.808006+0000","last_fullsized":"2024-04-21T10:30:57.808006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:58.891918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805799+0000","last_change":"2024-04-21T10:29:00.923613+0000","last_active":"2024-04-21T10:30:57.805799+0000","last_peered":"2024-04-21T10:30:57.805799+0000","last_clean":"2024-04-21T10:30:57.805799+0000","last_became_active":"2024-04-21T10:29:00.923001+0000","last_became_peered":"2024-04-21T10:29:00.923001+0000","last_unstale":"2024-04-21T10:30:57.805799+0000","last_undegraded":"2024-04-21T10:30:57.805799+0000","last_fullsized":"2024-04-21T10:30:57.805799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:44:53.593903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868537+0000","last_change":"2024-04-21T10:29:01.934974+0000","last_active":"2024-04-21T10:30:57.868537+0000","last_peered":"2024-04-21T10:30:57.868537+0000","last_clean":"2024-04-21T10:30:57.868537+0000","last_became_active":"2024-04-21T10:29:01.932443+0000","last_became_peered":"2024-04-21T10:29:01.932443+0000","last_unstale":"2024-04-21T10:30:57.868537+0000","last_undegraded":"2024-04-21T10:30:57.868537+0000","last_fullsized":"2024-04-21T10:30:57.868537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:24:47.190142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807475+0000","last_change":"2024-04-21T10:29:00.928179+0000","last_active":"2024-04-21T10:30:57.807475+0000","last_peered":"2024-04-21T10:30:57.807475+0000","last_clean":"2024-04-21T10:30:57.807475+0000","last_became_active":"2024-04-21T10:29:00.927588+0000","last_became_peered":"2024-04-21T10:29:00.927588+0000","last_unstale":"2024-04-21T10:30:57.807475+0000","last_undegraded":"2024-04-21T10:30:57.807475+0000","last_fullsized":"2024-04-21T10:30:57.807475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:04:41.081424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868724+0000","last_change":"2024-04-21T10:29:01.944421+0000","last_active":"2024-04-21T10:30:57.868724+0000","last_peered":"2024-04-21T10:30:57.868724+0000","last_clean":"2024-04-21T10:30:57.868724+0000","last_became_active":"2024-04-21T10:29:01.943572+0000","last_became_peered":"2024-04-21T10:29:01.943572+0000","last_unstale":"2024-04-21T10:30:57.868724+0000","last_undegraded":"2024-04-21T10:30:57.868724+0000","last_fullsized":"2024-04-21T10:30:57.868724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:34:01.690600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":138,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.973545+0000","last_change":"2024-04-21T10:29:00.924406+0000","last_active":"2024-04-21T10:30:57.973545+0000","last_peered":"2024-04-21T10:30:57.973545+0000","last_clean":"2024-04-21T10:30:57.973545+0000","last_became_active":"2024-04-21T10:29:00.924094+0000","last_became_peered":"2024-04-21T10:29:00.924094+0000","last_unstale":"2024-04-21T10:30:57.973545+0000","last_undegraded":"2024-04-21T10:30:57.973545+0000","last_fullsized":"2024-04-21T10:30:57.973545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:44.673751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807739+0000","last_change":"2024-04-21T10:29:01.933048+0000","last_active":"2024-04-21T10:30:57.807739+0000","last_peered":"2024-04-21T10:30:57.807739+0000","last_clean":"2024-04-21T10:30:57.807739+0000","last_became_active":"2024-04-21T10:29:01.932659+0000","last_became_peered":"2024-04-21T10:29:01.932659+0000","last_unstale":"2024-04-21T10:30:57.807739+0000","last_undegraded":"2024-04-21T10:30:57.807739+0000","last_fullsized":"2024-04-21T10:30:57.807739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:22:13.712112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869356+0000","last_change":"2024-04-21T10:29:00.923020+0000","last_active":"2024-04-21T10:30:57.869356+0000","last_peered":"2024-04-21T10:30:57.869356+0000","last_clean":"2024-04-21T10:30:57.869356+0000","last_became_active":"2024-04-21T10:29:00.922678+0000","last_became_peered":"2024-04-21T10:29:00.922678+0000","last_unstale":"2024-04-21T10:30:57.869356+0000","last_undegraded":"2024-04-21T10:30:57.869356+0000","last_fullsized":"2024-04-21T10:30:57.869356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:13:34.698082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869351+0000","last_change":"2024-04-21T10:29:01.947833+0000","last_active":"2024-04-21T10:30:57.869351+0000","last_peered":"2024-04-21T10:30:57.869351+0000","last_clean":"2024-04-21T10:30:57.869351+0000","last_became_active":"2024-04-21T10:29:01.947351+0000","last_became_peered":"2024-04-21T10:29:01.947351+0000","last_unstale":"2024-04-21T10:30:57.869351+0000","last_undegraded":"2024-04-21T10:30:57.869351+0000","last_fullsized":"2024-04-21T10:30:57.869351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:54.538731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807854+0000","last_change":"2024-04-21T10:29:00.932309+0000","last_active":"2024-04-21T10:30:57.807854+0000","last_peered":"2024-04-21T10:30:57.807854+0000","last_clean":"2024-04-21T10:30:57.807854+0000","last_became_active":"2024-04-21T10:29:00.931935+0000","last_became_peered":"2024-04-21T10:29:00.931935+0000","last_unstale":"2024-04-21T10:30:57.807854+0000","last_undegraded":"2024-04-21T10:30:57.807854+0000","last_fullsized":"2024-04-21T10:30:57.807854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:37:37.604381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807591+0000","last_change":"2024-04-21T10:29:01.946543+0000","last_active":"2024-04-21T10:30:57.807591+0000","last_peered":"2024-04-21T10:30:57.807591+0000","last_clean":"2024-04-21T10:30:57.807591+0000","last_became_active":"2024-04-21T10:29:01.946210+0000","last_became_peered":"2024-04-21T10:29:01.946210+0000","last_unstale":"2024-04-21T10:30:57.807591+0000","last_undegraded":"2024-04-21T10:30:57.807591+0000","last_fullsized":"2024-04-21T10:30:57.807591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:49.368851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'38","reported_seq":297,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.050618+0000","last_change":"2024-04-21T10:29:00.924109+0000","last_active":"2024-04-21T10:30:58.050618+0000","last_peered":"2024-04-21T10:30:58.050618+0000","last_clean":"2024-04-21T10:30:58.050618+0000","last_became_active":"2024-04-21T10:29:00.923735+0000","last_became_peered":"2024-04-21T10:29:00.923735+0000","last_unstale":"2024-04-21T10:30:58.050618+0000","last_undegraded":"2024-04-21T10:30:58.050618+0000","last_fullsized":"2024-04-21T10:30:58.050618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:13:54.015774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66735,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":238,"num_read_kb":184,"num_write":39,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869133+0000","last_change":"2024-04-21T10:29:01.937211+0000","last_active":"2024-04-21T10:30:57.869133+0000","last_peered":"2024-04-21T10:30:57.869133+0000","last_clean":"2024-04-21T10:30:57.869133+0000","last_became_active":"2024-04-21T10:29:01.936891+0000","last_became_peered":"2024-04-21T10:29:01.936891+0000","last_unstale":"2024-04-21T10:30:57.869133+0000","last_undegraded":"2024-04-21T10:30:57.869133+0000","last_fullsized":"2024-04-21T10:30:57.869133+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:26.062236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869231+0000","last_change":"2024-04-21T10:29:00.920626+0000","last_active":"2024-04-21T10:30:57.869231+0000","last_peered":"2024-04-21T10:30:57.869231+0000","last_clean":"2024-04-21T10:30:57.869231+0000","last_became_active":"2024-04-21T10:29:00.920317+0000","last_became_peered":"2024-04-21T10:29:00.920317+0000","last_unstale":"2024-04-21T10:30:57.869231+0000","last_undegraded":"2024-04-21T10:30:57.869231+0000","last_fullsized":"2024-04-21T10:30:57.869231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:41:18.373403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805778+0000","last_change":"2024-04-21T10:29:01.942494+0000","last_active":"2024-04-21T10:30:57.805778+0000","last_peered":"2024-04-21T10:30:57.805778+0000","last_clean":"2024-04-21T10:30:57.805778+0000","last_became_active":"2024-04-21T10:29:01.942151+0000","last_became_peered":"2024-04-21T10:29:01.942151+0000","last_unstale":"2024-04-21T10:30:57.805778+0000","last_undegraded":"2024-04-21T10:30:57.805778+0000","last_fullsized":"2024-04-21T10:30:57.805778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:39.071158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807798+0000","last_change":"2024-04-21T10:29:00.928221+0000","last_active":"2024-04-21T10:30:57.807798+0000","last_peered":"2024-04-21T10:30:57.807798+0000","last_clean":"2024-04-21T10:30:57.807798+0000","last_became_active":"2024-04-21T10:29:00.927873+0000","last_became_peered":"2024-04-21T10:29:00.927873+0000","last_unstale":"2024-04-21T10:30:57.807798+0000","last_undegraded":"2024-04-21T10:30:57.807798+0000","last_fullsized":"2024-04-21T10:30:57.807798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:14.898456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869676+0000","last_change":"2024-04-21T10:29:01.930375+0000","last_active":"2024-04-21T10:30:57.869676+0000","last_peered":"2024-04-21T10:30:57.869676+0000","last_clean":"2024-04-21T10:30:57.869676+0000","last_became_active":"2024-04-21T10:29:01.930009+0000","last_became_peered":"2024-04-21T10:29:01.930009+0000","last_unstale":"2024-04-21T10:30:57.869676+0000","last_undegraded":"2024-04-21T10:30:57.869676+0000","last_fullsized":"2024-04-21T10:30:57.869676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:34.111797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.053554+0000","last_change":"2024-04-21T10:29:00.922981+0000","last_active":"2024-04-21T10:30:58.053554+0000","last_peered":"2024-04-21T10:30:58.053554+0000","last_clean":"2024-04-21T10:30:58.053554+0000","last_became_active":"2024-04-21T10:29:00.922668+0000","last_became_peered":"2024-04-21T10:29:00.922668+0000","last_unstale":"2024-04-21T10:30:58.053554+0000","last_undegraded":"2024-04-21T10:30:58.053554+0000","last_fullsized":"2024-04-21T10:30:58.053554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:47.109375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868377+0000","last_change":"2024-04-21T10:29:01.942530+0000","last_active":"2024-04-21T10:30:57.868377+0000","last_peered":"2024-04-21T10:30:57.868377+0000","last_clean":"2024-04-21T10:30:57.868377+0000","last_became_active":"2024-04-21T10:29:01.942162+0000","last_became_peered":"2024-04-21T10:29:01.942162+0000","last_unstale":"2024-04-21T10:30:57.868377+0000","last_undegraded":"2024-04-21T10:30:57.868377+0000","last_fullsized":"2024-04-21T10:30:57.868377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:46:19.138930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868297+0000","last_change":"2024-04-21T10:30:03.544712+0000","last_active":"2024-04-21T10:30:57.868297+0000","last_peered":"2024-04-21T10:30:57.868297+0000","last_clean":"2024-04-21T10:30:57.868297+0000","last_became_active":"2024-04-21T10:30:03.519730+0000","last_became_peered":"2024-04-21T10:30:03.519730+0000","last_unstale":"2024-04-21T10:30:57.868297+0000","last_undegraded":"2024-04-21T10:30:57.868297+0000","last_fullsized":"2024-04-21T10:30:57.868297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:54:33.184653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807494+0000","last_change":"2024-04-21T10:29:01.929696+0000","last_active":"2024-04-21T10:30:57.807494+0000","last_peered":"2024-04-21T10:30:57.807494+0000","last_clean":"2024-04-21T10:30:57.807494+0000","last_became_active":"2024-04-21T10:29:01.929328+0000","last_became_peered":"2024-04-21T10:29:01.929328+0000","last_unstale":"2024-04-21T10:30:57.807494+0000","last_undegraded":"2024-04-21T10:30:57.807494+0000","last_fullsized":"2024-04-21T10:30:57.807494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:55:42.735595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869457+0000","last_change":"2024-04-21T10:29:00.925981+0000","last_active":"2024-04-21T10:30:57.869457+0000","last_peered":"2024-04-21T10:30:57.869457+0000","last_clean":"2024-04-21T10:30:57.869457+0000","last_became_active":"2024-04-21T10:29:00.925560+0000","last_became_peered":"2024-04-21T10:29:00.925560+0000","last_unstale":"2024-04-21T10:30:57.869457+0000","last_undegraded":"2024-04-21T10:30:57.869457+0000","last_fullsized":"2024-04-21T10:30:57.869457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:37.467522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868974+0000","last_change":"2024-04-21T10:29:01.941961+0000","last_active":"2024-04-21T10:30:57.868974+0000","last_peered":"2024-04-21T10:30:57.868974+0000","last_clean":"2024-04-21T10:30:57.868974+0000","last_became_active":"2024-04-21T10:29:01.941649+0000","last_became_peered":"2024-04-21T10:29:01.941649+0000","last_unstale":"2024-04-21T10:30:57.868974+0000","last_undegraded":"2024-04-21T10:30:57.868974+0000","last_fullsized":"2024-04-21T10:30:57.868974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:00:56.803856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869019+0000","last_change":"2024-04-21T10:29:00.929504+0000","last_active":"2024-04-21T10:30:57.869019+0000","last_peered":"2024-04-21T10:30:57.869019+0000","last_clean":"2024-04-21T10:30:57.869019+0000","last_became_active":"2024-04-21T10:29:00.929172+0000","last_became_peered":"2024-04-21T10:29:00.929172+0000","last_unstale":"2024-04-21T10:30:57.869019+0000","last_undegraded":"2024-04-21T10:30:57.869019+0000","last_fullsized":"2024-04-21T10:30:57.869019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:40.512175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869000+0000","last_change":"2024-04-21T10:29:01.939333+0000","last_active":"2024-04-21T10:30:57.869000+0000","last_peered":"2024-04-21T10:30:57.869000+0000","last_clean":"2024-04-21T10:30:57.869000+0000","last_became_active":"2024-04-21T10:29:01.938924+0000","last_became_peered":"2024-04-21T10:29:01.938924+0000","last_unstale":"2024-04-21T10:30:57.869000+0000","last_undegraded":"2024-04-21T10:30:57.869000+0000","last_fullsized":"2024-04-21T10:30:57.869000+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:20:01.287604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868977+0000","last_change":"2024-04-21T10:29:00.927455+0000","last_active":"2024-04-21T10:30:57.868977+0000","last_peered":"2024-04-21T10:30:57.868977+0000","last_clean":"2024-04-21T10:30:57.868977+0000","last_became_active":"2024-04-21T10:29:00.927180+0000","last_became_peered":"2024-04-21T10:29:00.927180+0000","last_unstale":"2024-04-21T10:30:57.868977+0000","last_undegraded":"2024-04-21T10:30:57.868977+0000","last_fullsized":"2024-04-21T10:30:57.868977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:11:57.166607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868482+0000","last_change":"2024-04-21T10:29:01.937646+0000","last_active":"2024-04-21T10:30:57.868482+0000","last_peered":"2024-04-21T10:30:57.868482+0000","last_clean":"2024-04-21T10:30:57.868482+0000","last_became_active":"2024-04-21T10:29:01.937216+0000","last_became_peered":"2024-04-21T10:29:01.937216+0000","last_unstale":"2024-04-21T10:30:57.868482+0000","last_undegraded":"2024-04-21T10:30:57.868482+0000","last_fullsized":"2024-04-21T10:30:57.868482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:38.265252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807620+0000","last_change":"2024-04-21T10:29:01.926860+0000","last_active":"2024-04-21T10:30:57.807620+0000","last_peered":"2024-04-21T10:30:57.807620+0000","last_clean":"2024-04-21T10:30:57.807620+0000","last_became_active":"2024-04-21T10:29:01.925951+0000","last_became_peered":"2024-04-21T10:29:01.925951+0000","last_unstale":"2024-04-21T10:30:57.807620+0000","last_undegraded":"2024-04-21T10:30:57.807620+0000","last_fullsized":"2024-04-21T10:30:57.807620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:01:55.127301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:31:02.979561+0000","last_change":"2024-04-21T10:29:00.926502+0000","last_active":"2024-04-21T10:31:02.979561+0000","last_peered":"2024-04-21T10:31:02.979561+0000","last_clean":"2024-04-21T10:31:02.979561+0000","last_became_active":"2024-04-21T10:29:00.926175+0000","last_became_peered":"2024-04-21T10:29:00.926175+0000","last_unstale":"2024-04-21T10:31:02.979561+0000","last_undegraded":"2024-04-21T10:31:02.979561+0000","last_fullsized":"2024-04-21T10:31:02.979561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:40:34.774085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868727+0000","last_change":"2024-04-21T10:30:03.519801+0000","last_active":"2024-04-21T10:30:57.868727+0000","last_peered":"2024-04-21T10:30:57.868727+0000","last_clean":"2024-04-21T10:30:57.868727+0000","last_became_active":"2024-04-21T10:30:03.519351+0000","last_became_peered":"2024-04-21T10:30:03.519351+0000","last_unstale":"2024-04-21T10:30:57.868727+0000","last_undegraded":"2024-04-21T10:30:57.868727+0000","last_fullsized":"2024-04-21T10:30:57.868727+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:13:36.675062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868748+0000","last_change":"2024-04-21T10:29:00.921748+0000","last_active":"2024-04-21T10:30:57.868748+0000","last_peered":"2024-04-21T10:30:57.868748+0000","last_clean":"2024-04-21T10:30:57.868748+0000","last_became_active":"2024-04-21T10:29:00.921408+0000","last_became_peered":"2024-04-21T10:29:00.921408+0000","last_unstale":"2024-04-21T10:30:57.868748+0000","last_undegraded":"2024-04-21T10:30:57.868748+0000","last_fullsized":"2024-04-21T10:30:57.868748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:52:27.606280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869101+0000","last_change":"2024-04-21T10:27:49.787009+0000","last_active":"2024-04-21T10:30:57.869101+0000","last_peered":"2024-04-21T10:30:57.869101+0000","last_clean":"2024-04-21T10:30:57.869101+0000","last_became_active":"2024-04-21T10:27:49.786530+0000","last_became_peered":"2024-04-21T10:27:49.786530+0000","last_unstale":"2024-04-21T10:30:57.869101+0000","last_undegraded":"2024-04-21T10:30:57.869101+0000","last_fullsized":"2024-04-21T10:30:57.869101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_clean_scrub_stamp":"2024-04-21T10:24:28.367927+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:20:12.106231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805706+0000","last_change":"2024-04-21T10:30:03.305720+0000","last_active":"2024-04-21T10:30:57.805706+0000","last_peered":"2024-04-21T10:30:57.805706+0000","last_clean":"2024-04-21T10:30:57.805706+0000","last_became_active":"2024-04-21T10:30:03.305368+0000","last_became_peered":"2024-04-21T10:30:03.305368+0000","last_unstale":"2024-04-21T10:30:57.805706+0000","last_undegraded":"2024-04-21T10:30:57.805706+0000","last_fullsized":"2024-04-21T10:30:57.805706+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:16.668743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869177+0000","last_change":"2024-04-21T10:29:00.927521+0000","last_active":"2024-04-21T10:30:57.869177+0000","last_peered":"2024-04-21T10:30:57.869177+0000","last_clean":"2024-04-21T10:30:57.869177+0000","last_became_active":"2024-04-21T10:29:00.927202+0000","last_became_peered":"2024-04-21T10:29:00.927202+0000","last_unstale":"2024-04-21T10:30:57.869177+0000","last_undegraded":"2024-04-21T10:30:57.869177+0000","last_fullsized":"2024-04-21T10:30:57.869177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:11:32.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869508+0000","last_change":"2024-04-21T10:29:00.930728+0000","last_active":"2024-04-21T10:30:57.869508+0000","last_peered":"2024-04-21T10:30:57.869508+0000","last_clean":"2024-04-21T10:30:57.869508+0000","last_became_active":"2024-04-21T10:29:00.930457+0000","last_became_peered":"2024-04-21T10:29:00.930457+0000","last_unstale":"2024-04-21T10:30:57.869508+0000","last_undegraded":"2024-04-21T10:30:57.869508+0000","last_fullsized":"2024-04-21T10:30:57.869508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:28.032954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.806062+0000","last_change":"2024-04-21T10:29:01.932278+0000","last_active":"2024-04-21T10:30:57.806062+0000","last_peered":"2024-04-21T10:30:57.806062+0000","last_clean":"2024-04-21T10:30:57.806062+0000","last_became_active":"2024-04-21T10:29:01.931774+0000","last_became_peered":"2024-04-21T10:29:01.931774+0000","last_unstale":"2024-04-21T10:30:57.806062+0000","last_undegraded":"2024-04-21T10:30:57.806062+0000","last_fullsized":"2024-04-21T10:30:57.806062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:34:25.858475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'44","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.008537+0000","last_change":"2024-04-21T10:29:00.926071+0000","last_active":"2024-04-21T10:30:58.008537+0000","last_peered":"2024-04-21T10:30:58.008537+0000","last_clean":"2024-04-21T10:30:58.008537+0000","last_became_active":"2024-04-21T10:29:00.925607+0000","last_became_peered":"2024-04-21T10:29:00.925607+0000","last_unstale":"2024-04-21T10:30:58.008537+0000","last_undegraded":"2024-04-21T10:30:58.008537+0000","last_fullsized":"2024-04-21T10:30:58.008537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:26.350652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":78,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869595+0000","last_change":"2024-04-21T10:29:01.941365+0000","last_active":"2024-04-21T10:30:57.869595+0000","last_peered":"2024-04-21T10:30:57.869595+0000","last_clean":"2024-04-21T10:30:57.869595+0000","last_became_active":"2024-04-21T10:29:01.940102+0000","last_became_peered":"2024-04-21T10:29:01.940102+0000","last_unstale":"2024-04-21T10:30:57.869595+0000","last_undegraded":"2024-04-21T10:30:57.869595+0000","last_fullsized":"2024-04-21T10:30:57.869595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:21.976855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869277+0000","last_change":"2024-04-21T10:29:00.924322+0000","last_active":"2024-04-21T10:30:57.869277+0000","last_peered":"2024-04-21T10:30:57.869277+0000","last_clean":"2024-04-21T10:30:57.869277+0000","last_became_active":"2024-04-21T10:29:00.924016+0000","last_became_peered":"2024-04-21T10:29:00.924016+0000","last_unstale":"2024-04-21T10:30:57.869277+0000","last_undegraded":"2024-04-21T10:30:57.869277+0000","last_fullsized":"2024-04-21T10:30:57.869277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:49:24.004951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868884+0000","last_change":"2024-04-21T10:29:01.946315+0000","last_active":"2024-04-21T10:30:57.868884+0000","last_peered":"2024-04-21T10:30:57.868884+0000","last_clean":"2024-04-21T10:30:57.868884+0000","last_became_active":"2024-04-21T10:29:01.945920+0000","last_became_peered":"2024-04-21T10:29:01.945920+0000","last_unstale":"2024-04-21T10:30:57.868884+0000","last_undegraded":"2024-04-21T10:30:57.868884+0000","last_fullsized":"2024-04-21T10:30:57.868884+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:10:52.361615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807342+0000","last_change":"2024-04-21T10:29:00.919086+0000","last_active":"2024-04-21T10:30:57.807342+0000","last_peered":"2024-04-21T10:30:57.807342+0000","last_clean":"2024-04-21T10:30:57.807342+0000","last_became_active":"2024-04-21T10:29:00.918727+0000","last_became_peered":"2024-04-21T10:29:00.918727+0000","last_unstale":"2024-04-21T10:30:57.807342+0000","last_undegraded":"2024-04-21T10:30:57.807342+0000","last_fullsized":"2024-04-21T10:30:57.807342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:53:27.209135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869231+0000","last_change":"2024-04-21T10:29:01.944037+0000","last_active":"2024-04-21T10:30:57.869231+0000","last_peered":"2024-04-21T10:30:57.869231+0000","last_clean":"2024-04-21T10:30:57.869231+0000","last_became_active":"2024-04-21T10:29:01.943692+0000","last_became_peered":"2024-04-21T10:29:01.943692+0000","last_unstale":"2024-04-21T10:30:57.869231+0000","last_undegraded":"2024-04-21T10:30:57.869231+0000","last_fullsized":"2024-04-21T10:30:57.869231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:51.285535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868942+0000","last_change":"2024-04-21T10:29:00.930937+0000","last_active":"2024-04-21T10:30:57.868942+0000","last_peered":"2024-04-21T10:30:57.868942+0000","last_clean":"2024-04-21T10:30:57.868942+0000","last_became_active":"2024-04-21T10:29:00.930348+0000","last_became_peered":"2024-04-21T10:29:00.930348+0000","last_unstale":"2024-04-21T10:30:57.868942+0000","last_undegraded":"2024-04-21T10:30:57.868942+0000","last_fullsized":"2024-04-21T10:30:57.868942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:43:33.801432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868384+0000","last_change":"2024-04-21T10:29:01.937116+0000","last_active":"2024-04-21T10:30:57.868384+0000","last_peered":"2024-04-21T10:30:57.868384+0000","last_clean":"2024-04-21T10:30:57.868384+0000","last_became_active":"2024-04-21T10:29:01.936756+0000","last_became_peered":"2024-04-21T10:29:01.936756+0000","last_unstale":"2024-04-21T10:30:57.868384+0000","last_undegraded":"2024-04-21T10:30:57.868384+0000","last_fullsized":"2024-04-21T10:30:57.868384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:02.732399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868588+0000","last_change":"2024-04-21T10:30:03.312469+0000","last_active":"2024-04-21T10:30:57.868588+0000","last_peered":"2024-04-21T10:30:57.868588+0000","last_clean":"2024-04-21T10:30:57.868588+0000","last_became_active":"2024-04-21T10:30:03.310027+0000","last_became_peered":"2024-04-21T10:30:03.310027+0000","last_unstale":"2024-04-21T10:30:57.868588+0000","last_undegraded":"2024-04-21T10:30:57.868588+0000","last_fullsized":"2024-04-21T10:30:57.868588+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:12:18.582539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869264+0000","last_change":"2024-04-21T10:29:01.940677+0000","last_active":"2024-04-21T10:30:57.869264+0000","last_peered":"2024-04-21T10:30:57.869264+0000","last_clean":"2024-04-21T10:30:57.869264+0000","last_became_active":"2024-04-21T10:29:01.940296+0000","last_became_peered":"2024-04-21T10:29:01.940296+0000","last_unstale":"2024-04-21T10:30:57.869264+0000","last_undegraded":"2024-04-21T10:30:57.869264+0000","last_fullsized":"2024-04-21T10:30:57.869264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:24:52.662041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868754+0000","last_change":"2024-04-21T10:29:00.924392+0000","last_active":"2024-04-21T10:30:57.868754+0000","last_peered":"2024-04-21T10:30:57.868754+0000","last_clean":"2024-04-21T10:30:57.868754+0000","last_became_active":"2024-04-21T10:29:00.924029+0000","last_became_peered":"2024-04-21T10:29:00.924029+0000","last_unstale":"2024-04-21T10:30:57.868754+0000","last_undegraded":"2024-04-21T10:30:57.868754+0000","last_fullsized":"2024-04-21T10:30:57.868754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:23:50.726978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869200+0000","last_change":"2024-04-21T10:29:01.939687+0000","last_active":"2024-04-21T10:30:57.869200+0000","last_peered":"2024-04-21T10:30:57.869200+0000","last_clean":"2024-04-21T10:30:57.869200+0000","last_became_active":"2024-04-21T10:29:01.939343+0000","last_became_peered":"2024-04-21T10:29:01.939343+0000","last_unstale":"2024-04-21T10:30:57.869200+0000","last_undegraded":"2024-04-21T10:30:57.869200+0000","last_fullsized":"2024-04-21T10:30:57.869200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:50.481998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869578+0000","last_change":"2024-04-21T10:29:00.923115+0000","last_active":"2024-04-21T10:30:57.869578+0000","last_peered":"2024-04-21T10:30:57.869578+0000","last_clean":"2024-04-21T10:30:57.869578+0000","last_became_active":"2024-04-21T10:29:00.922707+0000","last_became_peered":"2024-04-21T10:29:00.922707+0000","last_unstale":"2024-04-21T10:30:57.869578+0000","last_undegraded":"2024-04-21T10:30:57.869578+0000","last_fullsized":"2024-04-21T10:30:57.869578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:55:53.419904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869663+0000","last_change":"2024-04-21T10:29:01.940666+0000","last_active":"2024-04-21T10:30:57.869663+0000","last_peered":"2024-04-21T10:30:57.869663+0000","last_clean":"2024-04-21T10:30:57.869663+0000","last_became_active":"2024-04-21T10:29:01.940331+0000","last_became_peered":"2024-04-21T10:29:01.940331+0000","last_unstale":"2024-04-21T10:30:57.869663+0000","last_undegraded":"2024-04-21T10:30:57.869663+0000","last_fullsized":"2024-04-21T10:30:57.869663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:52:41.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'39","reported_seq":68,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.006266+0000","last_change":"2024-04-21T10:29:00.927873+0000","last_active":"2024-04-21T10:30:58.006266+0000","last_peered":"2024-04-21T10:30:58.006266+0000","last_clean":"2024-04-21T10:30:58.006266+0000","last_became_active":"2024-04-21T10:29:00.927403+0000","last_became_peered":"2024-04-21T10:29:00.927403+0000","last_unstale":"2024-04-21T10:30:58.006266+0000","last_undegraded":"2024-04-21T10:30:58.006266+0000","last_fullsized":"2024-04-21T10:30:58.006266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:05:05.276978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869023+0000","last_change":"2024-04-21T10:29:01.944725+0000","last_active":"2024-04-21T10:30:57.869023+0000","last_peered":"2024-04-21T10:30:57.869023+0000","last_clean":"2024-04-21T10:30:57.869023+0000","last_became_active":"2024-04-21T10:29:01.944326+0000","last_became_peered":"2024-04-21T10:29:01.944326+0000","last_unstale":"2024-04-21T10:30:57.869023+0000","last_undegraded":"2024-04-21T10:30:57.869023+0000","last_fullsized":"2024-04-21T10:30:57.869023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:50.209064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.806132+0000","last_change":"2024-04-21T10:29:00.924694+0000","last_active":"2024-04-21T10:30:57.806132+0000","last_peered":"2024-04-21T10:30:57.806132+0000","last_clean":"2024-04-21T10:30:57.806132+0000","last_became_active":"2024-04-21T10:29:00.924373+0000","last_became_peered":"2024-04-21T10:29:00.924373+0000","last_unstale":"2024-04-21T10:30:57.806132+0000","last_undegraded":"2024-04-21T10:30:57.806132+0000","last_fullsized":"2024-04-21T10:30:57.806132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:10:14.681630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869611+0000","last_change":"2024-04-21T10:29:01.939277+0000","last_active":"2024-04-21T10:30:57.869611+0000","last_peered":"2024-04-21T10:30:57.869611+0000","last_clean":"2024-04-21T10:30:57.869611+0000","last_became_active":"2024-04-21T10:29:01.938844+0000","last_became_peered":"2024-04-21T10:29:01.938844+0000","last_unstale":"2024-04-21T10:30:57.869611+0000","last_undegraded":"2024-04-21T10:30:57.869611+0000","last_fullsized":"2024-04-21T10:30:57.869611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:38:14.553868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868465+0000","last_change":"2024-04-21T10:29:01.942654+0000","last_active":"2024-04-21T10:30:57.868465+0000","last_peered":"2024-04-21T10:30:57.868465+0000","last_clean":"2024-04-21T10:30:57.868465+0000","last_became_active":"2024-04-21T10:29:01.942324+0000","last_became_peered":"2024-04-21T10:29:01.942324+0000","last_unstale":"2024-04-21T10:30:57.868465+0000","last_undegraded":"2024-04-21T10:30:57.868465+0000","last_fullsized":"2024-04-21T10:30:57.868465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:19.027138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868724+0000","last_change":"2024-04-21T10:30:03.307328+0000","last_active":"2024-04-21T10:30:57.868724+0000","last_peered":"2024-04-21T10:30:57.868724+0000","last_clean":"2024-04-21T10:30:57.868724+0000","last_became_active":"2024-04-21T10:30:03.306976+0000","last_became_peered":"2024-04-21T10:30:03.306976+0000","last_unstale":"2024-04-21T10:30:57.868724+0000","last_undegraded":"2024-04-21T10:30:57.868724+0000","last_fullsized":"2024-04-21T10:30:57.868724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:19:31.739372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868933+0000","last_change":"2024-04-21T10:29:01.944405+0000","last_active":"2024-04-21T10:30:57.868933+0000","last_peered":"2024-04-21T10:30:57.868933+0000","last_clean":"2024-04-21T10:30:57.868933+0000","last_became_active":"2024-04-21T10:29:01.944085+0000","last_became_peered":"2024-04-21T10:29:01.944085+0000","last_unstale":"2024-04-21T10:30:57.868933+0000","last_undegraded":"2024-04-21T10:30:57.868933+0000","last_fullsized":"2024-04-21T10:30:57.868933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:13.299859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868270+0000","last_change":"2024-04-21T10:29:01.941054+0000","last_active":"2024-04-21T10:30:57.868270+0000","last_peered":"2024-04-21T10:30:57.868270+0000","last_clean":"2024-04-21T10:30:57.868270+0000","last_became_active":"2024-04-21T10:29:01.940243+0000","last_became_peered":"2024-04-21T10:29:01.940243+0000","last_unstale":"2024-04-21T10:30:57.868270+0000","last_undegraded":"2024-04-21T10:30:57.868270+0000","last_fullsized":"2024-04-21T10:30:57.868270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:31:04.222026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805890+0000","last_change":"2024-04-21T10:30:03.307984+0000","last_active":"2024-04-21T10:30:57.805890+0000","last_peered":"2024-04-21T10:30:57.805890+0000","last_clean":"2024-04-21T10:30:57.805890+0000","last_became_active":"2024-04-21T10:30:03.307584+0000","last_became_peered":"2024-04-21T10:30:03.307584+0000","last_unstale":"2024-04-21T10:30:57.805890+0000","last_undegraded":"2024-04-21T10:30:57.805890+0000","last_fullsized":"2024-04-21T10:30:57.805890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:12.699665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868858+0000","last_change":"2024-04-21T10:29:01.947268+0000","last_active":"2024-04-21T10:30:57.868858+0000","last_peered":"2024-04-21T10:30:57.868858+0000","last_clean":"2024-04-21T10:30:57.868858+0000","last_became_active":"2024-04-21T10:29:01.946921+0000","last_became_peered":"2024-04-21T10:29:01.946921+0000","last_unstale":"2024-04-21T10:30:57.868858+0000","last_undegraded":"2024-04-21T10:30:57.868858+0000","last_fullsized":"2024-04-21T10:30:57.868858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:50:43.784920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869679+0000","last_change":"2024-04-21T10:29:01.949969+0000","last_active":"2024-04-21T10:30:57.869679+0000","last_peered":"2024-04-21T10:30:57.869679+0000","last_clean":"2024-04-21T10:30:57.869679+0000","last_became_active":"2024-04-21T10:29:01.949487+0000","last_became_peered":"2024-04-21T10:29:01.949487+0000","last_unstale":"2024-04-21T10:30:57.869679+0000","last_undegraded":"2024-04-21T10:30:57.869679+0000","last_fullsized":"2024-04-21T10:30:57.869679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:26:59.620209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869496+0000","last_change":"2024-04-21T10:29:01.937904+0000","last_active":"2024-04-21T10:30:57.869496+0000","last_peered":"2024-04-21T10:30:57.869496+0000","last_clean":"2024-04-21T10:30:57.869496+0000","last_became_active":"2024-04-21T10:29:01.937504+0000","last_became_peered":"2024-04-21T10:29:01.937504+0000","last_unstale":"2024-04-21T10:30:57.869496+0000","last_undegraded":"2024-04-21T10:30:57.869496+0000","last_fullsized":"2024-04-21T10:30:57.869496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:58:24.245118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807511+0000","last_change":"2024-04-21T10:30:03.310042+0000","last_active":"2024-04-21T10:30:57.807511+0000","last_peered":"2024-04-21T10:30:57.807511+0000","last_clean":"2024-04-21T10:30:57.807511+0000","last_became_active":"2024-04-21T10:30:03.309663+0000","last_became_peered":"2024-04-21T10:30:03.309663+0000","last_unstale":"2024-04-21T10:30:57.807511+0000","last_undegraded":"2024-04-21T10:30:57.807511+0000","last_fullsized":"2024-04-21T10:30:57.807511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:19.906949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868797+0000","last_change":"2024-04-21T10:29:01.936880+0000","last_active":"2024-04-21T10:30:57.868797+0000","last_peered":"2024-04-21T10:30:57.868797+0000","last_clean":"2024-04-21T10:30:57.868797+0000","last_became_active":"2024-04-21T10:29:01.936555+0000","last_became_peered":"2024-04-21T10:29:01.936555+0000","last_unstale":"2024-04-21T10:30:57.868797+0000","last_undegraded":"2024-04-21T10:30:57.868797+0000","last_fullsized":"2024-04-21T10:30:57.868797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:32.831606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807441+0000","last_change":"2024-04-21T10:29:01.943238+0000","last_active":"2024-04-21T10:30:57.807441+0000","last_peered":"2024-04-21T10:30:57.807441+0000","last_clean":"2024-04-21T10:30:57.807441+0000","last_became_active":"2024-04-21T10:29:01.942841+0000","last_became_peered":"2024-04-21T10:29:01.942841+0000","last_unstale":"2024-04-21T10:30:57.807441+0000","last_undegraded":"2024-04-21T10:30:57.807441+0000","last_fullsized":"2024-04-21T10:30:57.807441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:59:48.551524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805955+0000","last_change":"2024-04-21T10:29:01.933712+0000","last_active":"2024-04-21T10:30:57.805955+0000","last_peered":"2024-04-21T10:30:57.805955+0000","last_clean":"2024-04-21T10:30:57.805955+0000","last_became_active":"2024-04-21T10:29:01.933399+0000","last_became_peered":"2024-04-21T10:29:01.933399+0000","last_unstale":"2024-04-21T10:30:57.805955+0000","last_undegraded":"2024-04-21T10:30:57.805955+0000","last_fullsized":"2024-04-21T10:30:57.805955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:34:16.346403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868641+0000","last_change":"2024-04-21T10:29:01.947225+0000","last_active":"2024-04-21T10:30:57.868641+0000","last_peered":"2024-04-21T10:30:57.868641+0000","last_clean":"2024-04-21T10:30:57.868641+0000","last_became_active":"2024-04-21T10:29:01.946890+0000","last_became_peered":"2024-04-21T10:29:01.946890+0000","last_unstale":"2024-04-21T10:30:57.868641+0000","last_undegraded":"2024-04-21T10:30:57.868641+0000","last_fullsized":"2024-04-21T10:30:57.868641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:19:00.960767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597257,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":806,"num_read_kb":513,"num_write":286,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2224128,"data_stored":1988403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318309,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":472945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579945,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28800,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716352,"statfs":{"total":95995035648,"available":95965544448,"internally_reserved":0,"allocated":1826816,"data_stored":1263963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743470,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28384,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716768,"statfs":{"total":95995035648,"available":95965970432,"internally_reserved":0,"allocated":1396736,"data_stored":866285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005107,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28376,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716776,"statfs":{"total":95995035648,"available":95965978624,"internally_reserved":0,"allocated":1388544,"data_stored":866093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1066624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":472458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":539098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":473729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":541006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28768,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716384,"statfs":{"total":95995035648,"available":95965577216,"internally_reserved":0,"allocated":1732608,"data_stored":1194217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":995328,"data_stored":473427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1609728,"data_stored":1064212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":393842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T10:31:03.972 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-21T10:31:03.972 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T10:31:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.112+0000 7f3b4f09a640 1 -- 172.21.15.26:0/3194947236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b481021a0 msgr2=0x7f3b48102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.112+0000 7f3b4f09a640 1 --2- 172.21.15.26:0/3194947236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b481021a0 0x7f3b48102580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009900 tx=0x7f3b3c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.112+0000 7f3b4f09a640 1 -- 172.21.15.26:0/3194947236 shutdown_connections 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.112+0000 7f3b4f09a640 1 --2- 172.21.15.26:0/3194947236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b48107270 0x7f3b4810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.112+0000 7f3b4f09a640 1 --2- 172.21.15.26:0/3194947236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b48102b50 0x7f3b48106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.112+0000 7f3b4f09a640 1 --2- 172.21.15.26:0/3194947236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b481021a0 0x7f3b48102580 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.112+0000 7f3b4f09a640 1 -- 172.21.15.26:0/3194947236 >> 172.21.15.26:0/3194947236 conn(0x7f3b480793a0 msgr2=0x7f3b480797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.112+0000 7f3b4f09a640 1 -- 172.21.15.26:0/3194947236 shutdown_connections 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.112+0000 7f3b4f09a640 1 -- 172.21.15.26:0/3194947236 wait complete. 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b4f09a640 1 Processor -- start 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b4f09a640 1 -- start start 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b4f09a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b481021a0 0x7f3b48074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b4f09a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b48107270 0x7f3b48074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b4f09a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b48074cd0 0x7f3b481a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b4f09a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48112300 con 0x7f3b48107270 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b4f09a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48112180 con 0x7f3b48074cd0 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b4f09a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48112480 con 0x7f3b481021a0 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b48107270 0x7f3b48074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b48107270 0x7f3b48074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46452/0 (socket says 172.21.15.26:46452) 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b47fff640 1 -- 172.21.15.26:0/258297039 learned_addr learned my addr 172.21.15.26:0/258297039 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b4d637640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b48074cd0 0x7f3b481a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b47fff640 1 -- 172.21.15.26:0/258297039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b481021a0 msgr2=0x7f3b48074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b4ce36640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b481021a0 0x7f3b48074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b47fff640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b481021a0 0x7f3b48074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b47fff640 1 -- 172.21.15.26:0/258297039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b48074cd0 msgr2=0x7f3b481a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b47fff640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b48074cd0 0x7f3b481a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b47fff640 1 -- 172.21.15.26:0/258297039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b481a7130 con 0x7f3b48107270 2024-04-21T10:31:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b4ce36640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b481021a0 0x7f3b48074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:04.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.113+0000 7f3b47fff640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b48107270 0x7f3b48074790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400b450 tx=0x7f3b3400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:04.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.114+0000 7f3b45ffb640 1 -- 172.21.15.26:0/258297039 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34015020 con 0x7f3b48107270 2024-04-21T10:31:04.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.114+0000 7f3b45ffb640 1 -- 172.21.15.26:0/258297039 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b34011d40 con 0x7f3b48107270 2024-04-21T10:31:04.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.114+0000 7f3b4f09a640 1 -- 172.21.15.26:0/258297039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b481a7420 con 0x7f3b48107270 2024-04-21T10:31:04.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.114+0000 7f3b45ffb640 1 -- 172.21.15.26:0/258297039 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400f340 con 0x7f3b48107270 2024-04-21T10:31:04.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.114+0000 7f3b4f09a640 1 -- 172.21.15.26:0/258297039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b481a7970 con 0x7f3b48107270 2024-04-21T10:31:04.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.115+0000 7f3b4f09a640 1 -- 172.21.15.26:0/258297039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b48069230 con 0x7f3b48107270 2024-04-21T10:31:04.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.116+0000 7f3b45ffb640 1 -- 172.21.15.26:0/258297039 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f3b34018020 con 0x7f3b48107270 2024-04-21T10:31:04.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.116+0000 7f3b45ffb640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f3b380a3f00 0x7f3b380a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:04.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.116+0000 7f3b4ce36640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f3b380a3f00 0x7f3b380a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:04.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.116+0000 7f3b45ffb640 1 -- 172.21.15.26:0/258297039 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f3b340d04e0 con 0x7f3b48107270 2024-04-21T10:31:04.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.117+0000 7f3b4ce36640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f3b380a3f00 0x7f3b380a63c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c036040 tx=0x7f3b3c005fd0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:04.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.121+0000 7f3b45ffb640 1 -- 172.21.15.26:0/258297039 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b3401f020 con 0x7f3b48107270 2024-04-21T10:31:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.307+0000 7f3b4f09a640 1 -- 172.21.15.26:0/258297039 --> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b48102b50 con 0x7f3b380a3f00 2024-04-21T10:31:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.315+0000 7f3b45ffb640 1 -- 172.21.15.26:0/258297039 <== mgr.24869 v2:172.21.15.26:6800/4245046040 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257738 (secure 0 0 0) 0x7f3b48102b50 con 0x7f3b380a3f00 2024-04-21T10:31:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:04.318 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-21T10:31:04.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.318+0000 7f3b4f09a640 1 -- 172.21.15.26:0/258297039 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f3b380a3f00 msgr2=0x7f3b380a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.318+0000 7f3b4f09a640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f3b380a3f00 0x7f3b380a63c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c036040 tx=0x7f3b3c005fd0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.318+0000 7f3b4f09a640 1 -- 172.21.15.26:0/258297039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b48107270 msgr2=0x7f3b48074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.318+0000 7f3b4f09a640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b48107270 0x7f3b48074790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400b450 tx=0x7f3b3400b910 comp rx=0 tx=0).stop 2024-04-21T10:31:04.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.318+0000 7f3b4f09a640 1 -- 172.21.15.26:0/258297039 shutdown_connections 2024-04-21T10:31:04.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.318+0000 7f3b4f09a640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f3b380a3f00 0x7f3b380a63c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.318+0000 7f3b4f09a640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b48074cd0 0x7f3b481a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.318+0000 7f3b4f09a640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b48107270 0x7f3b48074790 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.318+0000 7f3b4f09a640 1 --2- 172.21.15.26:0/258297039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b481021a0 0x7f3b48074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.318+0000 7f3b4f09a640 1 -- 172.21.15.26:0/258297039 >> 172.21.15.26:0/258297039 conn(0x7f3b480793a0 msgr2=0x7f3b48100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:04.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.318+0000 7f3b4f09a640 1 -- 172.21.15.26:0/258297039 shutdown_connections 2024-04-21T10:31:04.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.318+0000 7f3b4f09a640 1 -- 172.21.15.26:0/258297039 wait complete. 2024-04-21T10:31:04.329 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-21T10:31:03.820090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187859,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":942,"num_read_kb":632,"num_write":469,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364572,"kb_used_data":15324,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124577252,"statfs":{"total":1151940427776,"available":1151567106048,"internally_reserved":0,"allocated":15691776,"data_stored":9294057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.989910"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869187+0000","last_change":"2024-04-21T10:29:01.940996+0000","last_active":"2024-04-21T10:30:57.869187+0000","last_peered":"2024-04-21T10:30:57.869187+0000","last_clean":"2024-04-21T10:30:57.869187+0000","last_became_active":"2024-04-21T10:29:01.940662+0000","last_became_peered":"2024-04-21T10:29:01.940662+0000","last_unstale":"2024-04-21T10:30:57.869187+0000","last_undegraded":"2024-04-21T10:30:57.869187+0000","last_fullsized":"2024-04-21T10:30:57.869187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:16:01.434895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868745+0000","last_change":"2024-04-21T10:29:01.939238+0000","last_active":"2024-04-21T10:30:57.868745+0000","last_peered":"2024-04-21T10:30:57.868745+0000","last_clean":"2024-04-21T10:30:57.868745+0000","last_became_active":"2024-04-21T10:29:01.938907+0000","last_became_peered":"2024-04-21T10:29:01.938907+0000","last_unstale":"2024-04-21T10:30:57.868745+0000","last_undegraded":"2024-04-21T10:30:57.868745+0000","last_fullsized":"2024-04-21T10:30:57.868745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:09.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869242+0000","last_change":"2024-04-21T10:29:01.932338+0000","last_active":"2024-04-21T10:30:57.869242+0000","last_peered":"2024-04-21T10:30:57.869242+0000","last_clean":"2024-04-21T10:30:57.869242+0000","last_became_active":"2024-04-21T10:29:01.931916+0000","last_became_peered":"2024-04-21T10:29:01.931916+0000","last_unstale":"2024-04-21T10:30:57.869242+0000","last_undegraded":"2024-04-21T10:30:57.869242+0000","last_fullsized":"2024-04-21T10:30:57.869242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:52.859695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869822+0000","last_change":"2024-04-21T10:29:01.945572+0000","last_active":"2024-04-21T10:30:57.869822+0000","last_peered":"2024-04-21T10:30:57.869822+0000","last_clean":"2024-04-21T10:30:57.869822+0000","last_became_active":"2024-04-21T10:29:01.940114+0000","last_became_peered":"2024-04-21T10:29:01.940114+0000","last_unstale":"2024-04-21T10:30:57.869822+0000","last_undegraded":"2024-04-21T10:30:57.869822+0000","last_fullsized":"2024-04-21T10:30:57.869822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:21.935573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868608+0000","last_change":"2024-04-21T10:29:01.946021+0000","last_active":"2024-04-21T10:30:57.868608+0000","last_peered":"2024-04-21T10:30:57.868608+0000","last_clean":"2024-04-21T10:30:57.868608+0000","last_became_active":"2024-04-21T10:29:01.945637+0000","last_became_peered":"2024-04-21T10:29:01.945637+0000","last_unstale":"2024-04-21T10:30:57.868608+0000","last_undegraded":"2024-04-21T10:30:57.868608+0000","last_fullsized":"2024-04-21T10:30:57.868608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:20.840494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868714+0000","last_change":"2024-04-21T10:29:01.935402+0000","last_active":"2024-04-21T10:30:57.868714+0000","last_peered":"2024-04-21T10:30:57.868714+0000","last_clean":"2024-04-21T10:30:57.868714+0000","last_became_active":"2024-04-21T10:29:01.934994+0000","last_became_peered":"2024-04-21T10:29:01.934994+0000","last_unstale":"2024-04-21T10:30:57.868714+0000","last_undegraded":"2024-04-21T10:30:57.868714+0000","last_fullsized":"2024-04-21T10:30:57.868714+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:50:18.320530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868977+0000","last_change":"2024-04-21T10:29:01.943185+0000","last_active":"2024-04-21T10:30:57.868977+0000","last_peered":"2024-04-21T10:30:57.868977+0000","last_clean":"2024-04-21T10:30:57.868977+0000","last_became_active":"2024-04-21T10:29:01.942805+0000","last_became_peered":"2024-04-21T10:29:01.942805+0000","last_unstale":"2024-04-21T10:30:57.868977+0000","last_undegraded":"2024-04-21T10:30:57.868977+0000","last_fullsized":"2024-04-21T10:30:57.868977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:06.776472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868710+0000","last_change":"2024-04-21T10:29:01.940995+0000","last_active":"2024-04-21T10:30:57.868710+0000","last_peered":"2024-04-21T10:30:57.868710+0000","last_clean":"2024-04-21T10:30:57.868710+0000","last_became_active":"2024-04-21T10:29:01.939173+0000","last_became_peered":"2024-04-21T10:29:01.939173+0000","last_unstale":"2024-04-21T10:30:57.868710+0000","last_undegraded":"2024-04-21T10:30:57.868710+0000","last_fullsized":"2024-04-21T10:30:57.868710+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:14:11.740147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807868+0000","last_change":"2024-04-21T10:29:01.944284+0000","last_active":"2024-04-21T10:30:57.807868+0000","last_peered":"2024-04-21T10:30:57.807868+0000","last_clean":"2024-04-21T10:30:57.807868+0000","last_became_active":"2024-04-21T10:29:01.943965+0000","last_became_peered":"2024-04-21T10:29:01.943965+0000","last_unstale":"2024-04-21T10:30:57.807868+0000","last_undegraded":"2024-04-21T10:30:57.807868+0000","last_fullsized":"2024-04-21T10:30:57.807868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:08:54.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869273+0000","last_change":"2024-04-21T10:29:01.936250+0000","last_active":"2024-04-21T10:30:57.869273+0000","last_peered":"2024-04-21T10:30:57.869273+0000","last_clean":"2024-04-21T10:30:57.869273+0000","last_became_active":"2024-04-21T10:29:01.935904+0000","last_became_peered":"2024-04-21T10:29:01.935904+0000","last_unstale":"2024-04-21T10:30:57.869273+0000","last_undegraded":"2024-04-21T10:30:57.869273+0000","last_fullsized":"2024-04-21T10:30:57.869273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:35.789987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869510+0000","last_change":"2024-04-21T10:29:01.946550+0000","last_active":"2024-04-21T10:30:57.869510+0000","last_peered":"2024-04-21T10:30:57.869510+0000","last_clean":"2024-04-21T10:30:57.869510+0000","last_became_active":"2024-04-21T10:29:01.946170+0000","last_became_peered":"2024-04-21T10:29:01.946170+0000","last_unstale":"2024-04-21T10:30:57.869510+0000","last_undegraded":"2024-04-21T10:30:57.869510+0000","last_fullsized":"2024-04-21T10:30:57.869510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:25:47.671821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869011+0000","last_change":"2024-04-21T10:29:01.946036+0000","last_active":"2024-04-21T10:30:57.869011+0000","last_peered":"2024-04-21T10:30:57.869011+0000","last_clean":"2024-04-21T10:30:57.869011+0000","last_became_active":"2024-04-21T10:29:01.940705+0000","last_became_peered":"2024-04-21T10:29:01.940705+0000","last_unstale":"2024-04-21T10:30:57.869011+0000","last_undegraded":"2024-04-21T10:30:57.869011+0000","last_fullsized":"2024-04-21T10:30:57.869011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:11:33.196059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805947+0000","last_change":"2024-04-21T10:29:01.940438+0000","last_active":"2024-04-21T10:30:57.805947+0000","last_peered":"2024-04-21T10:30:57.805947+0000","last_clean":"2024-04-21T10:30:57.805947+0000","last_became_active":"2024-04-21T10:29:01.940085+0000","last_became_peered":"2024-04-21T10:29:01.940085+0000","last_unstale":"2024-04-21T10:30:57.805947+0000","last_undegraded":"2024-04-21T10:30:57.805947+0000","last_fullsized":"2024-04-21T10:30:57.805947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:22:47.828858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868102+0000","last_change":"2024-04-21T10:29:01.941667+0000","last_active":"2024-04-21T10:30:57.868102+0000","last_peered":"2024-04-21T10:30:57.868102+0000","last_clean":"2024-04-21T10:30:57.868102+0000","last_became_active":"2024-04-21T10:29:01.940745+0000","last_became_peered":"2024-04-21T10:29:01.940745+0000","last_unstale":"2024-04-21T10:30:57.868102+0000","last_undegraded":"2024-04-21T10:30:57.868102+0000","last_fullsized":"2024-04-21T10:30:57.868102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:45:51.825780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.867997+0000","last_change":"2024-04-21T10:29:01.947771+0000","last_active":"2024-04-21T10:30:57.867997+0000","last_peered":"2024-04-21T10:30:57.867997+0000","last_clean":"2024-04-21T10:30:57.867997+0000","last_became_active":"2024-04-21T10:29:01.946818+0000","last_became_peered":"2024-04-21T10:29:01.946818+0000","last_unstale":"2024-04-21T10:30:57.867997+0000","last_undegraded":"2024-04-21T10:30:57.867997+0000","last_fullsized":"2024-04-21T10:30:57.867997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:18:42.159152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807344+0000","last_change":"2024-04-21T10:29:01.942499+0000","last_active":"2024-04-21T10:30:57.807344+0000","last_peered":"2024-04-21T10:30:57.807344+0000","last_clean":"2024-04-21T10:30:57.807344+0000","last_became_active":"2024-04-21T10:29:01.942136+0000","last_became_peered":"2024-04-21T10:29:01.942136+0000","last_unstale":"2024-04-21T10:30:57.807344+0000","last_undegraded":"2024-04-21T10:30:57.807344+0000","last_fullsized":"2024-04-21T10:30:57.807344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:34:11.822230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869190+0000","last_change":"2024-04-21T10:29:01.948822+0000","last_active":"2024-04-21T10:30:57.869190+0000","last_peered":"2024-04-21T10:30:57.869190+0000","last_clean":"2024-04-21T10:30:57.869190+0000","last_became_active":"2024-04-21T10:29:01.948492+0000","last_became_peered":"2024-04-21T10:29:01.948492+0000","last_unstale":"2024-04-21T10:30:57.869190+0000","last_undegraded":"2024-04-21T10:30:57.869190+0000","last_fullsized":"2024-04-21T10:30:57.869190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:06.992585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869064+0000","last_change":"2024-04-21T10:29:01.931265+0000","last_active":"2024-04-21T10:30:57.869064+0000","last_peered":"2024-04-21T10:30:57.869064+0000","last_clean":"2024-04-21T10:30:57.869064+0000","last_became_active":"2024-04-21T10:29:01.930877+0000","last_became_peered":"2024-04-21T10:29:01.930877+0000","last_unstale":"2024-04-21T10:30:57.869064+0000","last_undegraded":"2024-04-21T10:30:57.869064+0000","last_fullsized":"2024-04-21T10:30:57.869064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:49:00.951226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869590+0000","last_change":"2024-04-21T10:29:01.941718+0000","last_active":"2024-04-21T10:30:57.869590+0000","last_peered":"2024-04-21T10:30:57.869590+0000","last_clean":"2024-04-21T10:30:57.869590+0000","last_became_active":"2024-04-21T10:29:01.941386+0000","last_became_peered":"2024-04-21T10:29:01.941386+0000","last_unstale":"2024-04-21T10:30:57.869590+0000","last_undegraded":"2024-04-21T10:30:57.869590+0000","last_fullsized":"2024-04-21T10:30:57.869590+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:51:29.847961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869350+0000","last_change":"2024-04-21T10:29:00.923670+0000","last_active":"2024-04-21T10:30:57.869350+0000","last_peered":"2024-04-21T10:30:57.869350+0000","last_clean":"2024-04-21T10:30:57.869350+0000","last_became_active":"2024-04-21T10:29:00.923100+0000","last_became_peered":"2024-04-21T10:29:00.923100+0000","last_unstale":"2024-04-21T10:30:57.869350+0000","last_undegraded":"2024-04-21T10:30:57.869350+0000","last_fullsized":"2024-04-21T10:30:57.869350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:32:45.692961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869435+0000","last_change":"2024-04-21T10:29:01.938131+0000","last_active":"2024-04-21T10:30:57.869435+0000","last_peered":"2024-04-21T10:30:57.869435+0000","last_clean":"2024-04-21T10:30:57.869435+0000","last_became_active":"2024-04-21T10:29:01.937752+0000","last_became_peered":"2024-04-21T10:29:01.937752+0000","last_unstale":"2024-04-21T10:30:57.869435+0000","last_undegraded":"2024-04-21T10:30:57.869435+0000","last_fullsized":"2024-04-21T10:30:57.869435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:06:49.383554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869517+0000","last_change":"2024-04-21T10:29:00.929877+0000","last_active":"2024-04-21T10:30:57.869517+0000","last_peered":"2024-04-21T10:30:57.869517+0000","last_clean":"2024-04-21T10:30:57.869517+0000","last_became_active":"2024-04-21T10:29:00.929564+0000","last_became_peered":"2024-04-21T10:29:00.929564+0000","last_unstale":"2024-04-21T10:30:57.869517+0000","last_undegraded":"2024-04-21T10:30:57.869517+0000","last_fullsized":"2024-04-21T10:30:57.869517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:35.617533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869262+0000","last_change":"2024-04-21T10:29:01.942782+0000","last_active":"2024-04-21T10:30:57.869262+0000","last_peered":"2024-04-21T10:30:57.869262+0000","last_clean":"2024-04-21T10:30:57.869262+0000","last_became_active":"2024-04-21T10:29:01.940501+0000","last_became_peered":"2024-04-21T10:29:01.940501+0000","last_unstale":"2024-04-21T10:30:57.869262+0000","last_undegraded":"2024-04-21T10:30:57.869262+0000","last_fullsized":"2024-04-21T10:30:57.869262+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:04:03.431016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869465+0000","last_change":"2024-04-21T10:29:00.924395+0000","last_active":"2024-04-21T10:30:57.869465+0000","last_peered":"2024-04-21T10:30:57.869465+0000","last_clean":"2024-04-21T10:30:57.869465+0000","last_became_active":"2024-04-21T10:29:00.924045+0000","last_became_peered":"2024-04-21T10:29:00.924045+0000","last_unstale":"2024-04-21T10:30:57.869465+0000","last_undegraded":"2024-04-21T10:30:57.869465+0000","last_fullsized":"2024-04-21T10:30:57.869465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:08:53.922213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807373+0000","last_change":"2024-04-21T10:29:01.934508+0000","last_active":"2024-04-21T10:30:57.807373+0000","last_peered":"2024-04-21T10:30:57.807373+0000","last_clean":"2024-04-21T10:30:57.807373+0000","last_became_active":"2024-04-21T10:29:01.934051+0000","last_became_peered":"2024-04-21T10:29:01.934051+0000","last_unstale":"2024-04-21T10:30:57.807373+0000","last_undegraded":"2024-04-21T10:30:57.807373+0000","last_fullsized":"2024-04-21T10:30:57.807373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:49:27.686603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868924+0000","last_change":"2024-04-21T10:29:00.925850+0000","last_active":"2024-04-21T10:30:57.868924+0000","last_peered":"2024-04-21T10:30:57.868924+0000","last_clean":"2024-04-21T10:30:57.868924+0000","last_became_active":"2024-04-21T10:29:00.925521+0000","last_became_peered":"2024-04-21T10:29:00.925521+0000","last_unstale":"2024-04-21T10:30:57.868924+0000","last_undegraded":"2024-04-21T10:30:57.868924+0000","last_fullsized":"2024-04-21T10:30:57.868924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:36:55.403413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807733+0000","last_change":"2024-04-21T10:29:01.930046+0000","last_active":"2024-04-21T10:30:57.807733+0000","last_peered":"2024-04-21T10:30:57.807733+0000","last_clean":"2024-04-21T10:30:57.807733+0000","last_became_active":"2024-04-21T10:29:01.929736+0000","last_became_peered":"2024-04-21T10:29:01.929736+0000","last_unstale":"2024-04-21T10:30:57.807733+0000","last_undegraded":"2024-04-21T10:30:57.807733+0000","last_fullsized":"2024-04-21T10:30:57.807733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:11:10.833267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":143,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.047873+0000","last_change":"2024-04-21T10:29:00.927936+0000","last_active":"2024-04-21T10:30:58.047873+0000","last_peered":"2024-04-21T10:30:58.047873+0000","last_clean":"2024-04-21T10:30:58.047873+0000","last_became_active":"2024-04-21T10:29:00.927352+0000","last_became_peered":"2024-04-21T10:29:00.927352+0000","last_unstale":"2024-04-21T10:30:58.047873+0000","last_undegraded":"2024-04-21T10:30:58.047873+0000","last_fullsized":"2024-04-21T10:30:58.047873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:14:10.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869207+0000","last_change":"2024-04-21T10:30:03.306538+0000","last_active":"2024-04-21T10:30:57.869207+0000","last_peered":"2024-04-21T10:30:57.869207+0000","last_clean":"2024-04-21T10:30:57.869207+0000","last_became_active":"2024-04-21T10:30:03.306197+0000","last_became_peered":"2024-04-21T10:30:03.306197+0000","last_unstale":"2024-04-21T10:30:57.869207+0000","last_undegraded":"2024-04-21T10:30:57.869207+0000","last_fullsized":"2024-04-21T10:30:57.869207+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:28:25.688821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'3","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868668+0000","last_change":"2024-04-21T10:30:03.522106+0000","last_active":"2024-04-21T10:30:57.868668+0000","last_peered":"2024-04-21T10:30:57.868668+0000","last_clean":"2024-04-21T10:30:57.868668+0000","last_became_active":"2024-04-21T10:30:03.519189+0000","last_became_peered":"2024-04-21T10:30:03.519189+0000","last_unstale":"2024-04-21T10:30:57.868668+0000","last_undegraded":"2024-04-21T10:30:57.868668+0000","last_fullsized":"2024-04-21T10:30:57.868668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:49:39.930818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.808006+0000","last_change":"2024-04-21T10:29:01.918627+0000","last_active":"2024-04-21T10:30:57.808006+0000","last_peered":"2024-04-21T10:30:57.808006+0000","last_clean":"2024-04-21T10:30:57.808006+0000","last_became_active":"2024-04-21T10:29:01.917993+0000","last_became_peered":"2024-04-21T10:29:01.917993+0000","last_unstale":"2024-04-21T10:30:57.808006+0000","last_undegraded":"2024-04-21T10:30:57.808006+0000","last_fullsized":"2024-04-21T10:30:57.808006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:58.891918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805799+0000","last_change":"2024-04-21T10:29:00.923613+0000","last_active":"2024-04-21T10:30:57.805799+0000","last_peered":"2024-04-21T10:30:57.805799+0000","last_clean":"2024-04-21T10:30:57.805799+0000","last_became_active":"2024-04-21T10:29:00.923001+0000","last_became_peered":"2024-04-21T10:29:00.923001+0000","last_unstale":"2024-04-21T10:30:57.805799+0000","last_undegraded":"2024-04-21T10:30:57.805799+0000","last_fullsized":"2024-04-21T10:30:57.805799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:44:53.593903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868537+0000","last_change":"2024-04-21T10:29:01.934974+0000","last_active":"2024-04-21T10:30:57.868537+0000","last_peered":"2024-04-21T10:30:57.868537+0000","last_clean":"2024-04-21T10:30:57.868537+0000","last_became_active":"2024-04-21T10:29:01.932443+0000","last_became_peered":"2024-04-21T10:29:01.932443+0000","last_unstale":"2024-04-21T10:30:57.868537+0000","last_undegraded":"2024-04-21T10:30:57.868537+0000","last_fullsized":"2024-04-21T10:30:57.868537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:24:47.190142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807475+0000","last_change":"2024-04-21T10:29:00.928179+0000","last_active":"2024-04-21T10:30:57.807475+0000","last_peered":"2024-04-21T10:30:57.807475+0000","last_clean":"2024-04-21T10:30:57.807475+0000","last_became_active":"2024-04-21T10:29:00.927588+0000","last_became_peered":"2024-04-21T10:29:00.927588+0000","last_unstale":"2024-04-21T10:30:57.807475+0000","last_undegraded":"2024-04-21T10:30:57.807475+0000","last_fullsized":"2024-04-21T10:30:57.807475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:04:41.081424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868724+0000","last_change":"2024-04-21T10:29:01.944421+0000","last_active":"2024-04-21T10:30:57.868724+0000","last_peered":"2024-04-21T10:30:57.868724+0000","last_clean":"2024-04-21T10:30:57.868724+0000","last_became_active":"2024-04-21T10:29:01.943572+0000","last_became_peered":"2024-04-21T10:29:01.943572+0000","last_unstale":"2024-04-21T10:30:57.868724+0000","last_undegraded":"2024-04-21T10:30:57.868724+0000","last_fullsized":"2024-04-21T10:30:57.868724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:34:01.690600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":138,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.973545+0000","last_change":"2024-04-21T10:29:00.924406+0000","last_active":"2024-04-21T10:30:57.973545+0000","last_peered":"2024-04-21T10:30:57.973545+0000","last_clean":"2024-04-21T10:30:57.973545+0000","last_became_active":"2024-04-21T10:29:00.924094+0000","last_became_peered":"2024-04-21T10:29:00.924094+0000","last_unstale":"2024-04-21T10:30:57.973545+0000","last_undegraded":"2024-04-21T10:30:57.973545+0000","last_fullsized":"2024-04-21T10:30:57.973545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:44.673751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807739+0000","last_change":"2024-04-21T10:29:01.933048+0000","last_active":"2024-04-21T10:30:57.807739+0000","last_peered":"2024-04-21T10:30:57.807739+0000","last_clean":"2024-04-21T10:30:57.807739+0000","last_became_active":"2024-04-21T10:29:01.932659+0000","last_became_peered":"2024-04-21T10:29:01.932659+0000","last_unstale":"2024-04-21T10:30:57.807739+0000","last_undegraded":"2024-04-21T10:30:57.807739+0000","last_fullsized":"2024-04-21T10:30:57.807739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:22:13.712112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869356+0000","last_change":"2024-04-21T10:29:00.923020+0000","last_active":"2024-04-21T10:30:57.869356+0000","last_peered":"2024-04-21T10:30:57.869356+0000","last_clean":"2024-04-21T10:30:57.869356+0000","last_became_active":"2024-04-21T10:29:00.922678+0000","last_became_peered":"2024-04-21T10:29:00.922678+0000","last_unstale":"2024-04-21T10:30:57.869356+0000","last_undegraded":"2024-04-21T10:30:57.869356+0000","last_fullsized":"2024-04-21T10:30:57.869356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:13:34.698082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869351+0000","last_change":"2024-04-21T10:29:01.947833+0000","last_active":"2024-04-21T10:30:57.869351+0000","last_peered":"2024-04-21T10:30:57.869351+0000","last_clean":"2024-04-21T10:30:57.869351+0000","last_became_active":"2024-04-21T10:29:01.947351+0000","last_became_peered":"2024-04-21T10:29:01.947351+0000","last_unstale":"2024-04-21T10:30:57.869351+0000","last_undegraded":"2024-04-21T10:30:57.869351+0000","last_fullsized":"2024-04-21T10:30:57.869351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:54.538731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807854+0000","last_change":"2024-04-21T10:29:00.932309+0000","last_active":"2024-04-21T10:30:57.807854+0000","last_peered":"2024-04-21T10:30:57.807854+0000","last_clean":"2024-04-21T10:30:57.807854+0000","last_became_active":"2024-04-21T10:29:00.931935+0000","last_became_peered":"2024-04-21T10:29:00.931935+0000","last_unstale":"2024-04-21T10:30:57.807854+0000","last_undegraded":"2024-04-21T10:30:57.807854+0000","last_fullsized":"2024-04-21T10:30:57.807854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:37:37.604381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807591+0000","last_change":"2024-04-21T10:29:01.946543+0000","last_active":"2024-04-21T10:30:57.807591+0000","last_peered":"2024-04-21T10:30:57.807591+0000","last_clean":"2024-04-21T10:30:57.807591+0000","last_became_active":"2024-04-21T10:29:01.946210+0000","last_became_peered":"2024-04-21T10:29:01.946210+0000","last_unstale":"2024-04-21T10:30:57.807591+0000","last_undegraded":"2024-04-21T10:30:57.807591+0000","last_fullsized":"2024-04-21T10:30:57.807591+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:34:49.368851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'38","reported_seq":297,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.050618+0000","last_change":"2024-04-21T10:29:00.924109+0000","last_active":"2024-04-21T10:30:58.050618+0000","last_peered":"2024-04-21T10:30:58.050618+0000","last_clean":"2024-04-21T10:30:58.050618+0000","last_became_active":"2024-04-21T10:29:00.923735+0000","last_became_peered":"2024-04-21T10:29:00.923735+0000","last_unstale":"2024-04-21T10:30:58.050618+0000","last_undegraded":"2024-04-21T10:30:58.050618+0000","last_fullsized":"2024-04-21T10:30:58.050618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:13:54.015774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66735,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":238,"num_read_kb":184,"num_write":39,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869133+0000","last_change":"2024-04-21T10:29:01.937211+0000","last_active":"2024-04-21T10:30:57.869133+0000","last_peered":"2024-04-21T10:30:57.869133+0000","last_clean":"2024-04-21T10:30:57.869133+0000","last_became_active":"2024-04-21T10:29:01.936891+0000","last_became_peered":"2024-04-21T10:29:01.936891+0000","last_unstale":"2024-04-21T10:30:57.869133+0000","last_undegraded":"2024-04-21T10:30:57.869133+0000","last_fullsized":"2024-04-21T10:30:57.869133+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:26.062236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'6","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869231+0000","last_change":"2024-04-21T10:29:00.920626+0000","last_active":"2024-04-21T10:30:57.869231+0000","last_peered":"2024-04-21T10:30:57.869231+0000","last_clean":"2024-04-21T10:30:57.869231+0000","last_became_active":"2024-04-21T10:29:00.920317+0000","last_became_peered":"2024-04-21T10:29:00.920317+0000","last_unstale":"2024-04-21T10:30:57.869231+0000","last_undegraded":"2024-04-21T10:30:57.869231+0000","last_fullsized":"2024-04-21T10:30:57.869231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:41:18.373403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805778+0000","last_change":"2024-04-21T10:29:01.942494+0000","last_active":"2024-04-21T10:30:57.805778+0000","last_peered":"2024-04-21T10:30:57.805778+0000","last_clean":"2024-04-21T10:30:57.805778+0000","last_became_active":"2024-04-21T10:29:01.942151+0000","last_became_peered":"2024-04-21T10:29:01.942151+0000","last_unstale":"2024-04-21T10:30:57.805778+0000","last_undegraded":"2024-04-21T10:30:57.805778+0000","last_fullsized":"2024-04-21T10:30:57.805778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:39.071158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807798+0000","last_change":"2024-04-21T10:29:00.928221+0000","last_active":"2024-04-21T10:30:57.807798+0000","last_peered":"2024-04-21T10:30:57.807798+0000","last_clean":"2024-04-21T10:30:57.807798+0000","last_became_active":"2024-04-21T10:29:00.927873+0000","last_became_peered":"2024-04-21T10:29:00.927873+0000","last_unstale":"2024-04-21T10:30:57.807798+0000","last_undegraded":"2024-04-21T10:30:57.807798+0000","last_fullsized":"2024-04-21T10:30:57.807798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:14.898456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869676+0000","last_change":"2024-04-21T10:29:01.930375+0000","last_active":"2024-04-21T10:30:57.869676+0000","last_peered":"2024-04-21T10:30:57.869676+0000","last_clean":"2024-04-21T10:30:57.869676+0000","last_became_active":"2024-04-21T10:29:01.930009+0000","last_became_peered":"2024-04-21T10:29:01.930009+0000","last_unstale":"2024-04-21T10:30:57.869676+0000","last_undegraded":"2024-04-21T10:30:57.869676+0000","last_fullsized":"2024-04-21T10:30:57.869676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:05:34.111797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.053554+0000","last_change":"2024-04-21T10:29:00.922981+0000","last_active":"2024-04-21T10:30:58.053554+0000","last_peered":"2024-04-21T10:30:58.053554+0000","last_clean":"2024-04-21T10:30:58.053554+0000","last_became_active":"2024-04-21T10:29:00.922668+0000","last_became_peered":"2024-04-21T10:29:00.922668+0000","last_unstale":"2024-04-21T10:30:58.053554+0000","last_undegraded":"2024-04-21T10:30:58.053554+0000","last_fullsized":"2024-04-21T10:30:58.053554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:33:47.109375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868377+0000","last_change":"2024-04-21T10:29:01.942530+0000","last_active":"2024-04-21T10:30:57.868377+0000","last_peered":"2024-04-21T10:30:57.868377+0000","last_clean":"2024-04-21T10:30:57.868377+0000","last_became_active":"2024-04-21T10:29:01.942162+0000","last_became_peered":"2024-04-21T10:29:01.942162+0000","last_unstale":"2024-04-21T10:30:57.868377+0000","last_undegraded":"2024-04-21T10:30:57.868377+0000","last_fullsized":"2024-04-21T10:30:57.868377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:46:19.138930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868297+0000","last_change":"2024-04-21T10:30:03.544712+0000","last_active":"2024-04-21T10:30:57.868297+0000","last_peered":"2024-04-21T10:30:57.868297+0000","last_clean":"2024-04-21T10:30:57.868297+0000","last_became_active":"2024-04-21T10:30:03.519730+0000","last_became_peered":"2024-04-21T10:30:03.519730+0000","last_unstale":"2024-04-21T10:30:57.868297+0000","last_undegraded":"2024-04-21T10:30:57.868297+0000","last_fullsized":"2024-04-21T10:30:57.868297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:54:33.184653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807494+0000","last_change":"2024-04-21T10:29:01.929696+0000","last_active":"2024-04-21T10:30:57.807494+0000","last_peered":"2024-04-21T10:30:57.807494+0000","last_clean":"2024-04-21T10:30:57.807494+0000","last_became_active":"2024-04-21T10:29:01.929328+0000","last_became_peered":"2024-04-21T10:29:01.929328+0000","last_unstale":"2024-04-21T10:30:57.807494+0000","last_undegraded":"2024-04-21T10:30:57.807494+0000","last_fullsized":"2024-04-21T10:30:57.807494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:55:42.735595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869457+0000","last_change":"2024-04-21T10:29:00.925981+0000","last_active":"2024-04-21T10:30:57.869457+0000","last_peered":"2024-04-21T10:30:57.869457+0000","last_clean":"2024-04-21T10:30:57.869457+0000","last_became_active":"2024-04-21T10:29:00.925560+0000","last_became_peered":"2024-04-21T10:29:00.925560+0000","last_unstale":"2024-04-21T10:30:57.869457+0000","last_undegraded":"2024-04-21T10:30:57.869457+0000","last_fullsized":"2024-04-21T10:30:57.869457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:12:37.467522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868974+0000","last_change":"2024-04-21T10:29:01.941961+0000","last_active":"2024-04-21T10:30:57.868974+0000","last_peered":"2024-04-21T10:30:57.868974+0000","last_clean":"2024-04-21T10:30:57.868974+0000","last_became_active":"2024-04-21T10:29:01.941649+0000","last_became_peered":"2024-04-21T10:29:01.941649+0000","last_unstale":"2024-04-21T10:30:57.868974+0000","last_undegraded":"2024-04-21T10:30:57.868974+0000","last_fullsized":"2024-04-21T10:30:57.868974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:00:56.803856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869019+0000","last_change":"2024-04-21T10:29:00.929504+0000","last_active":"2024-04-21T10:30:57.869019+0000","last_peered":"2024-04-21T10:30:57.869019+0000","last_clean":"2024-04-21T10:30:57.869019+0000","last_became_active":"2024-04-21T10:29:00.929172+0000","last_became_peered":"2024-04-21T10:29:00.929172+0000","last_unstale":"2024-04-21T10:30:57.869019+0000","last_undegraded":"2024-04-21T10:30:57.869019+0000","last_fullsized":"2024-04-21T10:30:57.869019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:40.512175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869000+0000","last_change":"2024-04-21T10:29:01.939333+0000","last_active":"2024-04-21T10:30:57.869000+0000","last_peered":"2024-04-21T10:30:57.869000+0000","last_clean":"2024-04-21T10:30:57.869000+0000","last_became_active":"2024-04-21T10:29:01.938924+0000","last_became_peered":"2024-04-21T10:29:01.938924+0000","last_unstale":"2024-04-21T10:30:57.869000+0000","last_undegraded":"2024-04-21T10:30:57.869000+0000","last_fullsized":"2024-04-21T10:30:57.869000+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:20:01.287604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868977+0000","last_change":"2024-04-21T10:29:00.927455+0000","last_active":"2024-04-21T10:30:57.868977+0000","last_peered":"2024-04-21T10:30:57.868977+0000","last_clean":"2024-04-21T10:30:57.868977+0000","last_became_active":"2024-04-21T10:29:00.927180+0000","last_became_peered":"2024-04-21T10:29:00.927180+0000","last_unstale":"2024-04-21T10:30:57.868977+0000","last_undegraded":"2024-04-21T10:30:57.868977+0000","last_fullsized":"2024-04-21T10:30:57.868977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:11:57.166607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868482+0000","last_change":"2024-04-21T10:29:01.937646+0000","last_active":"2024-04-21T10:30:57.868482+0000","last_peered":"2024-04-21T10:30:57.868482+0000","last_clean":"2024-04-21T10:30:57.868482+0000","last_became_active":"2024-04-21T10:29:01.937216+0000","last_became_peered":"2024-04-21T10:29:01.937216+0000","last_unstale":"2024-04-21T10:30:57.868482+0000","last_undegraded":"2024-04-21T10:30:57.868482+0000","last_fullsized":"2024-04-21T10:30:57.868482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:38.265252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807620+0000","last_change":"2024-04-21T10:29:01.926860+0000","last_active":"2024-04-21T10:30:57.807620+0000","last_peered":"2024-04-21T10:30:57.807620+0000","last_clean":"2024-04-21T10:30:57.807620+0000","last_became_active":"2024-04-21T10:29:01.925951+0000","last_became_peered":"2024-04-21T10:29:01.925951+0000","last_unstale":"2024-04-21T10:30:57.807620+0000","last_undegraded":"2024-04-21T10:30:57.807620+0000","last_fullsized":"2024-04-21T10:30:57.807620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:01:55.127301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:31:02.979561+0000","last_change":"2024-04-21T10:29:00.926502+0000","last_active":"2024-04-21T10:31:02.979561+0000","last_peered":"2024-04-21T10:31:02.979561+0000","last_clean":"2024-04-21T10:31:02.979561+0000","last_became_active":"2024-04-21T10:29:00.926175+0000","last_became_peered":"2024-04-21T10:29:00.926175+0000","last_unstale":"2024-04-21T10:31:02.979561+0000","last_undegraded":"2024-04-21T10:31:02.979561+0000","last_fullsized":"2024-04-21T10:31:02.979561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:40:34.774085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868727+0000","last_change":"2024-04-21T10:30:03.519801+0000","last_active":"2024-04-21T10:30:57.868727+0000","last_peered":"2024-04-21T10:30:57.868727+0000","last_clean":"2024-04-21T10:30:57.868727+0000","last_became_active":"2024-04-21T10:30:03.519351+0000","last_became_peered":"2024-04-21T10:30:03.519351+0000","last_unstale":"2024-04-21T10:30:57.868727+0000","last_undegraded":"2024-04-21T10:30:57.868727+0000","last_fullsized":"2024-04-21T10:30:57.868727+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:13:36.675062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"87'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868748+0000","last_change":"2024-04-21T10:29:00.921748+0000","last_active":"2024-04-21T10:30:57.868748+0000","last_peered":"2024-04-21T10:30:57.868748+0000","last_clean":"2024-04-21T10:30:57.868748+0000","last_became_active":"2024-04-21T10:29:00.921408+0000","last_became_peered":"2024-04-21T10:29:00.921408+0000","last_unstale":"2024-04-21T10:30:57.868748+0000","last_undegraded":"2024-04-21T10:30:57.868748+0000","last_fullsized":"2024-04-21T10:30:57.868748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:52:27.606280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869101+0000","last_change":"2024-04-21T10:27:49.787009+0000","last_active":"2024-04-21T10:30:57.869101+0000","last_peered":"2024-04-21T10:30:57.869101+0000","last_clean":"2024-04-21T10:30:57.869101+0000","last_became_active":"2024-04-21T10:27:49.786530+0000","last_became_peered":"2024-04-21T10:27:49.786530+0000","last_unstale":"2024-04-21T10:30:57.869101+0000","last_undegraded":"2024-04-21T10:30:57.869101+0000","last_fullsized":"2024-04-21T10:30:57.869101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:24:28.367927+0000","last_clean_scrub_stamp":"2024-04-21T10:24:28.367927+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:20:12.106231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805706+0000","last_change":"2024-04-21T10:30:03.305720+0000","last_active":"2024-04-21T10:30:57.805706+0000","last_peered":"2024-04-21T10:30:57.805706+0000","last_clean":"2024-04-21T10:30:57.805706+0000","last_became_active":"2024-04-21T10:30:03.305368+0000","last_became_peered":"2024-04-21T10:30:03.305368+0000","last_unstale":"2024-04-21T10:30:57.805706+0000","last_undegraded":"2024-04-21T10:30:57.805706+0000","last_fullsized":"2024-04-21T10:30:57.805706+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:16.668743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869177+0000","last_change":"2024-04-21T10:29:00.927521+0000","last_active":"2024-04-21T10:30:57.869177+0000","last_peered":"2024-04-21T10:30:57.869177+0000","last_clean":"2024-04-21T10:30:57.869177+0000","last_became_active":"2024-04-21T10:29:00.927202+0000","last_became_peered":"2024-04-21T10:29:00.927202+0000","last_unstale":"2024-04-21T10:30:57.869177+0000","last_undegraded":"2024-04-21T10:30:57.869177+0000","last_fullsized":"2024-04-21T10:30:57.869177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:11:32.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869508+0000","last_change":"2024-04-21T10:29:00.930728+0000","last_active":"2024-04-21T10:30:57.869508+0000","last_peered":"2024-04-21T10:30:57.869508+0000","last_clean":"2024-04-21T10:30:57.869508+0000","last_became_active":"2024-04-21T10:29:00.930457+0000","last_became_peered":"2024-04-21T10:29:00.930457+0000","last_unstale":"2024-04-21T10:30:57.869508+0000","last_undegraded":"2024-04-21T10:30:57.869508+0000","last_fullsized":"2024-04-21T10:30:57.869508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:28.032954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.806062+0000","last_change":"2024-04-21T10:29:01.932278+0000","last_active":"2024-04-21T10:30:57.806062+0000","last_peered":"2024-04-21T10:30:57.806062+0000","last_clean":"2024-04-21T10:30:57.806062+0000","last_became_active":"2024-04-21T10:29:01.931774+0000","last_became_peered":"2024-04-21T10:29:01.931774+0000","last_unstale":"2024-04-21T10:30:57.806062+0000","last_undegraded":"2024-04-21T10:30:57.806062+0000","last_fullsized":"2024-04-21T10:30:57.806062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:34:25.858475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'44","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.008537+0000","last_change":"2024-04-21T10:29:00.926071+0000","last_active":"2024-04-21T10:30:58.008537+0000","last_peered":"2024-04-21T10:30:58.008537+0000","last_clean":"2024-04-21T10:30:58.008537+0000","last_became_active":"2024-04-21T10:29:00.925607+0000","last_became_peered":"2024-04-21T10:29:00.925607+0000","last_unstale":"2024-04-21T10:30:58.008537+0000","last_undegraded":"2024-04-21T10:30:58.008537+0000","last_fullsized":"2024-04-21T10:30:58.008537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:23:26.350652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":78,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869595+0000","last_change":"2024-04-21T10:29:01.941365+0000","last_active":"2024-04-21T10:30:57.869595+0000","last_peered":"2024-04-21T10:30:57.869595+0000","last_clean":"2024-04-21T10:30:57.869595+0000","last_became_active":"2024-04-21T10:29:01.940102+0000","last_became_peered":"2024-04-21T10:29:01.940102+0000","last_unstale":"2024-04-21T10:30:57.869595+0000","last_undegraded":"2024-04-21T10:30:57.869595+0000","last_fullsized":"2024-04-21T10:30:57.869595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:12:21.976855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869277+0000","last_change":"2024-04-21T10:29:00.924322+0000","last_active":"2024-04-21T10:30:57.869277+0000","last_peered":"2024-04-21T10:30:57.869277+0000","last_clean":"2024-04-21T10:30:57.869277+0000","last_became_active":"2024-04-21T10:29:00.924016+0000","last_became_peered":"2024-04-21T10:29:00.924016+0000","last_unstale":"2024-04-21T10:30:57.869277+0000","last_undegraded":"2024-04-21T10:30:57.869277+0000","last_fullsized":"2024-04-21T10:30:57.869277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:49:24.004951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868884+0000","last_change":"2024-04-21T10:29:01.946315+0000","last_active":"2024-04-21T10:30:57.868884+0000","last_peered":"2024-04-21T10:30:57.868884+0000","last_clean":"2024-04-21T10:30:57.868884+0000","last_became_active":"2024-04-21T10:29:01.945920+0000","last_became_peered":"2024-04-21T10:29:01.945920+0000","last_unstale":"2024-04-21T10:30:57.868884+0000","last_undegraded":"2024-04-21T10:30:57.868884+0000","last_fullsized":"2024-04-21T10:30:57.868884+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:10:52.361615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807342+0000","last_change":"2024-04-21T10:29:00.919086+0000","last_active":"2024-04-21T10:30:57.807342+0000","last_peered":"2024-04-21T10:30:57.807342+0000","last_clean":"2024-04-21T10:30:57.807342+0000","last_became_active":"2024-04-21T10:29:00.918727+0000","last_became_peered":"2024-04-21T10:29:00.918727+0000","last_unstale":"2024-04-21T10:30:57.807342+0000","last_undegraded":"2024-04-21T10:30:57.807342+0000","last_fullsized":"2024-04-21T10:30:57.807342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:53:27.209135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869231+0000","last_change":"2024-04-21T10:29:01.944037+0000","last_active":"2024-04-21T10:30:57.869231+0000","last_peered":"2024-04-21T10:30:57.869231+0000","last_clean":"2024-04-21T10:30:57.869231+0000","last_became_active":"2024-04-21T10:29:01.943692+0000","last_became_peered":"2024-04-21T10:29:01.943692+0000","last_unstale":"2024-04-21T10:30:57.869231+0000","last_undegraded":"2024-04-21T10:30:57.869231+0000","last_fullsized":"2024-04-21T10:30:57.869231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:42:51.285535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868942+0000","last_change":"2024-04-21T10:29:00.930937+0000","last_active":"2024-04-21T10:30:57.868942+0000","last_peered":"2024-04-21T10:30:57.868942+0000","last_clean":"2024-04-21T10:30:57.868942+0000","last_became_active":"2024-04-21T10:29:00.930348+0000","last_became_peered":"2024-04-21T10:29:00.930348+0000","last_unstale":"2024-04-21T10:30:57.868942+0000","last_undegraded":"2024-04-21T10:30:57.868942+0000","last_fullsized":"2024-04-21T10:30:57.868942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:43:33.801432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868384+0000","last_change":"2024-04-21T10:29:01.937116+0000","last_active":"2024-04-21T10:30:57.868384+0000","last_peered":"2024-04-21T10:30:57.868384+0000","last_clean":"2024-04-21T10:30:57.868384+0000","last_became_active":"2024-04-21T10:29:01.936756+0000","last_became_peered":"2024-04-21T10:29:01.936756+0000","last_unstale":"2024-04-21T10:30:57.868384+0000","last_undegraded":"2024-04-21T10:30:57.868384+0000","last_fullsized":"2024-04-21T10:30:57.868384+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:02.732399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'1","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868588+0000","last_change":"2024-04-21T10:30:03.312469+0000","last_active":"2024-04-21T10:30:57.868588+0000","last_peered":"2024-04-21T10:30:57.868588+0000","last_clean":"2024-04-21T10:30:57.868588+0000","last_became_active":"2024-04-21T10:30:03.310027+0000","last_became_peered":"2024-04-21T10:30:03.310027+0000","last_unstale":"2024-04-21T10:30:57.868588+0000","last_undegraded":"2024-04-21T10:30:57.868588+0000","last_fullsized":"2024-04-21T10:30:57.868588+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T21:12:18.582539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869264+0000","last_change":"2024-04-21T10:29:01.940677+0000","last_active":"2024-04-21T10:30:57.869264+0000","last_peered":"2024-04-21T10:30:57.869264+0000","last_clean":"2024-04-21T10:30:57.869264+0000","last_became_active":"2024-04-21T10:29:01.940296+0000","last_became_peered":"2024-04-21T10:29:01.940296+0000","last_unstale":"2024-04-21T10:30:57.869264+0000","last_undegraded":"2024-04-21T10:30:57.869264+0000","last_fullsized":"2024-04-21T10:30:57.869264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:24:52.662041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868754+0000","last_change":"2024-04-21T10:29:00.924392+0000","last_active":"2024-04-21T10:30:57.868754+0000","last_peered":"2024-04-21T10:30:57.868754+0000","last_clean":"2024-04-21T10:30:57.868754+0000","last_became_active":"2024-04-21T10:29:00.924029+0000","last_became_peered":"2024-04-21T10:29:00.924029+0000","last_unstale":"2024-04-21T10:30:57.868754+0000","last_undegraded":"2024-04-21T10:30:57.868754+0000","last_fullsized":"2024-04-21T10:30:57.868754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:23:50.726978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869200+0000","last_change":"2024-04-21T10:29:01.939687+0000","last_active":"2024-04-21T10:30:57.869200+0000","last_peered":"2024-04-21T10:30:57.869200+0000","last_clean":"2024-04-21T10:30:57.869200+0000","last_became_active":"2024-04-21T10:29:01.939343+0000","last_became_peered":"2024-04-21T10:29:01.939343+0000","last_unstale":"2024-04-21T10:30:57.869200+0000","last_undegraded":"2024-04-21T10:30:57.869200+0000","last_fullsized":"2024-04-21T10:30:57.869200+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:54:50.481998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869578+0000","last_change":"2024-04-21T10:29:00.923115+0000","last_active":"2024-04-21T10:30:57.869578+0000","last_peered":"2024-04-21T10:30:57.869578+0000","last_clean":"2024-04-21T10:30:57.869578+0000","last_became_active":"2024-04-21T10:29:00.922707+0000","last_became_peered":"2024-04-21T10:29:00.922707+0000","last_unstale":"2024-04-21T10:30:57.869578+0000","last_undegraded":"2024-04-21T10:30:57.869578+0000","last_fullsized":"2024-04-21T10:30:57.869578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:55:53.419904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869663+0000","last_change":"2024-04-21T10:29:01.940666+0000","last_active":"2024-04-21T10:30:57.869663+0000","last_peered":"2024-04-21T10:30:57.869663+0000","last_clean":"2024-04-21T10:30:57.869663+0000","last_became_active":"2024-04-21T10:29:01.940331+0000","last_became_peered":"2024-04-21T10:29:01.940331+0000","last_unstale":"2024-04-21T10:30:57.869663+0000","last_undegraded":"2024-04-21T10:30:57.869663+0000","last_fullsized":"2024-04-21T10:30:57.869663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:52:41.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'39","reported_seq":68,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:58.006266+0000","last_change":"2024-04-21T10:29:00.927873+0000","last_active":"2024-04-21T10:30:58.006266+0000","last_peered":"2024-04-21T10:30:58.006266+0000","last_clean":"2024-04-21T10:30:58.006266+0000","last_became_active":"2024-04-21T10:29:00.927403+0000","last_became_peered":"2024-04-21T10:29:00.927403+0000","last_unstale":"2024-04-21T10:30:58.006266+0000","last_undegraded":"2024-04-21T10:30:58.006266+0000","last_fullsized":"2024-04-21T10:30:58.006266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:05:05.276978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869023+0000","last_change":"2024-04-21T10:29:01.944725+0000","last_active":"2024-04-21T10:30:57.869023+0000","last_peered":"2024-04-21T10:30:57.869023+0000","last_clean":"2024-04-21T10:30:57.869023+0000","last_became_active":"2024-04-21T10:29:01.944326+0000","last_became_peered":"2024-04-21T10:29:01.944326+0000","last_unstale":"2024-04-21T10:30:57.869023+0000","last_undegraded":"2024-04-21T10:30:57.869023+0000","last_fullsized":"2024-04-21T10:30:57.869023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:32:50.209064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.806132+0000","last_change":"2024-04-21T10:29:00.924694+0000","last_active":"2024-04-21T10:30:57.806132+0000","last_peered":"2024-04-21T10:30:57.806132+0000","last_clean":"2024-04-21T10:30:57.806132+0000","last_became_active":"2024-04-21T10:29:00.924373+0000","last_became_peered":"2024-04-21T10:29:00.924373+0000","last_unstale":"2024-04-21T10:30:57.806132+0000","last_undegraded":"2024-04-21T10:30:57.806132+0000","last_fullsized":"2024-04-21T10:30:57.806132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:28:59.902142+0000","last_clean_scrub_stamp":"2024-04-21T10:28:59.902142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:10:14.681630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869611+0000","last_change":"2024-04-21T10:29:01.939277+0000","last_active":"2024-04-21T10:30:57.869611+0000","last_peered":"2024-04-21T10:30:57.869611+0000","last_clean":"2024-04-21T10:30:57.869611+0000","last_became_active":"2024-04-21T10:29:01.938844+0000","last_became_peered":"2024-04-21T10:29:01.938844+0000","last_unstale":"2024-04-21T10:30:57.869611+0000","last_undegraded":"2024-04-21T10:30:57.869611+0000","last_fullsized":"2024-04-21T10:30:57.869611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:38:14.553868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868465+0000","last_change":"2024-04-21T10:29:01.942654+0000","last_active":"2024-04-21T10:30:57.868465+0000","last_peered":"2024-04-21T10:30:57.868465+0000","last_clean":"2024-04-21T10:30:57.868465+0000","last_became_active":"2024-04-21T10:29:01.942324+0000","last_became_peered":"2024-04-21T10:29:01.942324+0000","last_unstale":"2024-04-21T10:30:57.868465+0000","last_undegraded":"2024-04-21T10:30:57.868465+0000","last_fullsized":"2024-04-21T10:30:57.868465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:19.027138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868724+0000","last_change":"2024-04-21T10:30:03.307328+0000","last_active":"2024-04-21T10:30:57.868724+0000","last_peered":"2024-04-21T10:30:57.868724+0000","last_clean":"2024-04-21T10:30:57.868724+0000","last_became_active":"2024-04-21T10:30:03.306976+0000","last_became_peered":"2024-04-21T10:30:03.306976+0000","last_unstale":"2024-04-21T10:30:57.868724+0000","last_undegraded":"2024-04-21T10:30:57.868724+0000","last_fullsized":"2024-04-21T10:30:57.868724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T22:19:31.739372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868933+0000","last_change":"2024-04-21T10:29:01.944405+0000","last_active":"2024-04-21T10:30:57.868933+0000","last_peered":"2024-04-21T10:30:57.868933+0000","last_clean":"2024-04-21T10:30:57.868933+0000","last_became_active":"2024-04-21T10:29:01.944085+0000","last_became_peered":"2024-04-21T10:29:01.944085+0000","last_unstale":"2024-04-21T10:30:57.868933+0000","last_undegraded":"2024-04-21T10:30:57.868933+0000","last_fullsized":"2024-04-21T10:30:57.868933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:01:13.299859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868270+0000","last_change":"2024-04-21T10:29:01.941054+0000","last_active":"2024-04-21T10:30:57.868270+0000","last_peered":"2024-04-21T10:30:57.868270+0000","last_clean":"2024-04-21T10:30:57.868270+0000","last_became_active":"2024-04-21T10:29:01.940243+0000","last_became_peered":"2024-04-21T10:29:01.940243+0000","last_unstale":"2024-04-21T10:30:57.868270+0000","last_undegraded":"2024-04-21T10:30:57.868270+0000","last_fullsized":"2024-04-21T10:30:57.868270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:31:04.222026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805890+0000","last_change":"2024-04-21T10:30:03.307984+0000","last_active":"2024-04-21T10:30:57.805890+0000","last_peered":"2024-04-21T10:30:57.805890+0000","last_clean":"2024-04-21T10:30:57.805890+0000","last_became_active":"2024-04-21T10:30:03.307584+0000","last_became_peered":"2024-04-21T10:30:03.307584+0000","last_unstale":"2024-04-21T10:30:57.805890+0000","last_undegraded":"2024-04-21T10:30:57.805890+0000","last_fullsized":"2024-04-21T10:30:57.805890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:21:12.699665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868858+0000","last_change":"2024-04-21T10:29:01.947268+0000","last_active":"2024-04-21T10:30:57.868858+0000","last_peered":"2024-04-21T10:30:57.868858+0000","last_clean":"2024-04-21T10:30:57.868858+0000","last_became_active":"2024-04-21T10:29:01.946921+0000","last_became_peered":"2024-04-21T10:29:01.946921+0000","last_unstale":"2024-04-21T10:30:57.868858+0000","last_undegraded":"2024-04-21T10:30:57.868858+0000","last_fullsized":"2024-04-21T10:30:57.868858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:50:43.784920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869679+0000","last_change":"2024-04-21T10:29:01.949969+0000","last_active":"2024-04-21T10:30:57.869679+0000","last_peered":"2024-04-21T10:30:57.869679+0000","last_clean":"2024-04-21T10:30:57.869679+0000","last_became_active":"2024-04-21T10:29:01.949487+0000","last_became_peered":"2024-04-21T10:29:01.949487+0000","last_unstale":"2024-04-21T10:30:57.869679+0000","last_undegraded":"2024-04-21T10:30:57.869679+0000","last_fullsized":"2024-04-21T10:30:57.869679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:26:59.620209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.869496+0000","last_change":"2024-04-21T10:29:01.937904+0000","last_active":"2024-04-21T10:30:57.869496+0000","last_peered":"2024-04-21T10:30:57.869496+0000","last_clean":"2024-04-21T10:30:57.869496+0000","last_became_active":"2024-04-21T10:29:01.937504+0000","last_became_peered":"2024-04-21T10:29:01.937504+0000","last_unstale":"2024-04-21T10:30:57.869496+0000","last_undegraded":"2024-04-21T10:30:57.869496+0000","last_fullsized":"2024-04-21T10:30:57.869496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:58:24.245118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807511+0000","last_change":"2024-04-21T10:30:03.310042+0000","last_active":"2024-04-21T10:30:57.807511+0000","last_peered":"2024-04-21T10:30:57.807511+0000","last_clean":"2024-04-21T10:30:57.807511+0000","last_became_active":"2024-04-21T10:30:03.309663+0000","last_became_peered":"2024-04-21T10:30:03.309663+0000","last_unstale":"2024-04-21T10:30:57.807511+0000","last_undegraded":"2024-04-21T10:30:57.807511+0000","last_fullsized":"2024-04-21T10:30:57.807511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:19.906949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868797+0000","last_change":"2024-04-21T10:29:01.936880+0000","last_active":"2024-04-21T10:30:57.868797+0000","last_peered":"2024-04-21T10:30:57.868797+0000","last_clean":"2024-04-21T10:30:57.868797+0000","last_became_active":"2024-04-21T10:29:01.936555+0000","last_became_peered":"2024-04-21T10:29:01.936555+0000","last_unstale":"2024-04-21T10:30:57.868797+0000","last_undegraded":"2024-04-21T10:30:57.868797+0000","last_fullsized":"2024-04-21T10:30:57.868797+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:32.831606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.807441+0000","last_change":"2024-04-21T10:29:01.943238+0000","last_active":"2024-04-21T10:30:57.807441+0000","last_peered":"2024-04-21T10:30:57.807441+0000","last_clean":"2024-04-21T10:30:57.807441+0000","last_became_active":"2024-04-21T10:29:01.942841+0000","last_became_peered":"2024-04-21T10:29:01.942841+0000","last_unstale":"2024-04-21T10:30:57.807441+0000","last_undegraded":"2024-04-21T10:30:57.807441+0000","last_fullsized":"2024-04-21T10:30:57.807441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:59:48.551524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.805955+0000","last_change":"2024-04-21T10:29:01.933712+0000","last_active":"2024-04-21T10:30:57.805955+0000","last_peered":"2024-04-21T10:30:57.805955+0000","last_clean":"2024-04-21T10:30:57.805955+0000","last_became_active":"2024-04-21T10:29:01.933399+0000","last_became_peered":"2024-04-21T10:29:01.933399+0000","last_unstale":"2024-04-21T10:30:57.805955+0000","last_undegraded":"2024-04-21T10:30:57.805955+0000","last_fullsized":"2024-04-21T10:30:57.805955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:34:16.346403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-21T10:30:57.868641+0000","last_change":"2024-04-21T10:29:01.947225+0000","last_active":"2024-04-21T10:30:57.868641+0000","last_peered":"2024-04-21T10:30:57.868641+0000","last_clean":"2024-04-21T10:30:57.868641+0000","last_became_active":"2024-04-21T10:29:01.946890+0000","last_became_peered":"2024-04-21T10:29:01.946890+0000","last_unstale":"2024-04-21T10:30:57.868641+0000","last_undegraded":"2024-04-21T10:30:57.868641+0000","last_fullsized":"2024-04-21T10:30:57.868641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T10:29:00.904179+0000","last_clean_scrub_stamp":"2024-04-21T10:29:00.904179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:19:00.960767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597257,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":806,"num_read_kb":513,"num_write":286,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2224128,"data_stored":1988403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318309,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":472945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579945,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28800,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716352,"statfs":{"total":95995035648,"available":95965544448,"internally_reserved":0,"allocated":1826816,"data_stored":1263963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743470,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28384,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716768,"statfs":{"total":95995035648,"available":95965970432,"internally_reserved":0,"allocated":1396736,"data_stored":866285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005107,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28376,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716776,"statfs":{"total":95995035648,"available":95965978624,"internally_reserved":0,"allocated":1388544,"data_stored":866093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1066624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":472458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":539098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":473729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":541006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28768,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716384,"statfs":{"total":95995035648,"available":95965577216,"internally_reserved":0,"allocated":1732608,"data_stored":1194217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":995328,"data_stored":473427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1609728,"data_stored":1064212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":393842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T10:31:04.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:31:04.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.472+0000 7f44d1043640 1 -- 172.21.15.26:0/301244726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44cc101b60 msgr2=0x7f44cc10dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.472+0000 7f44d1043640 1 --2- 172.21.15.26:0/301244726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44cc101b60 0x7f44cc10dc90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009cd0 tx=0x7f44bc02c0f0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.472+0000 7f44d1043640 1 -- 172.21.15.26:0/301244726 shutdown_connections 2024-04-21T10:31:04.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.472+0000 7f44d1043640 1 --2- 172.21.15.26:0/301244726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44cc101b60 0x7f44cc10dc90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.472+0000 7f44d1043640 1 --2- 172.21.15.26:0/301244726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44cc1011a0 0x7f44cc101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.472+0000 7f44d1043640 1 --2- 172.21.15.26:0/301244726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44cc105e30 0x7f44cc106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.472+0000 7f44d1043640 1 -- 172.21.15.26:0/301244726 >> 172.21.15.26:0/301244726 conn(0x7f44cc077e30 msgr2=0x7f44cc078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:04.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.472+0000 7f44d1043640 1 -- 172.21.15.26:0/301244726 shutdown_connections 2024-04-21T10:31:04.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.472+0000 7f44d1043640 1 -- 172.21.15.26:0/301244726 wait complete. 2024-04-21T10:31:04.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.473+0000 7f44d1043640 1 Processor -- start 2024-04-21T10:31:04.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.473+0000 7f44d1043640 1 -- start start 2024-04-21T10:31:04.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.473+0000 7f44d1043640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44cc1011a0 0x7f44cc0feb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:04.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.473+0000 7f44d1043640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44cc101b60 0x7f44cc0ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:04.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.473+0000 7f44d1043640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44cc105e30 0x7f44cc0ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:04.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.473+0000 7f44d1043640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc1110e0 con 0x7f44cc1011a0 2024-04-21T10:31:04.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.473+0000 7f44d1043640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc110f60 con 0x7f44cc101b60 2024-04-21T10:31:04.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.473+0000 7f44d1043640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc111260 con 0x7f44cc105e30 2024-04-21T10:31:04.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44cad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44cc1011a0 0x7f44cc0feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44cad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44cc1011a0 0x7f44cc0feb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46472/0 (socket says 172.21.15.26:46472) 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44cad9d640 1 -- 172.21.15.26:0/2012462687 learned_addr learned my addr 172.21.15.26:0/2012462687 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44ca59c640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44cc101b60 0x7f44cc0ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44cb59e640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44cc105e30 0x7f44cc0ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44cad9d640 1 -- 172.21.15.26:0/2012462687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44cc105e30 msgr2=0x7f44cc0ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44cad9d640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44cc105e30 0x7f44cc0ff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44cad9d640 1 -- 172.21.15.26:0/2012462687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44cc101b60 msgr2=0x7f44cc0ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44cad9d640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44cc101b60 0x7f44cc0ff0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44cad9d640 1 -- 172.21.15.26:0/2012462687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44cc1a6c40 con 0x7f44cc1011a0 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44ca59c640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44cc101b60 0x7f44cc0ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44cad9d640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44cc1011a0 0x7f44cc0feb80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f44c400d950 tx=0x7f44c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44b3fff640 1 -- 172.21.15.26:0/2012462687 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c4017070 con 0x7f44cc1011a0 2024-04-21T10:31:04.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44d1043640 1 -- 172.21.15.26:0/2012462687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44cc1a6f30 con 0x7f44cc1011a0 2024-04-21T10:31:04.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44b3fff640 1 -- 172.21.15.26:0/2012462687 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44c4009c30 con 0x7f44cc1011a0 2024-04-21T10:31:04.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44b3fff640 1 -- 172.21.15.26:0/2012462687 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c400f620 con 0x7f44cc1011a0 2024-04-21T10:31:04.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.474+0000 7f44d1043640 1 -- 172.21.15.26:0/2012462687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44cc1a7440 con 0x7f44cc1011a0 2024-04-21T10:31:04.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.475+0000 7f44d1043640 1 -- 172.21.15.26:0/2012462687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44cc068de0 con 0x7f44cc1011a0 2024-04-21T10:31:04.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.476+0000 7f44b3fff640 1 -- 172.21.15.26:0/2012462687 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f44c4010460 con 0x7f44cc1011a0 2024-04-21T10:31:04.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.477+0000 7f44b3fff640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f44b40a3ed0 0x7f44b40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:04.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.477+0000 7f44ca59c640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f44b40a3ed0 0x7f44b40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:04.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.477+0000 7f44b3fff640 1 -- 172.21.15.26:0/2012462687 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f44c40d0740 con 0x7f44cc1011a0 2024-04-21T10:31:04.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.478+0000 7f44ca59c640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f44b40a3ed0 0x7f44b40a6390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f44c0005fd0 tx=0x7f44c0005ea0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:04.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.481+0000 7f44b3fff640 1 -- 172.21.15.26:0/2012462687 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44c409cbb0 con 0x7f44cc1011a0 2024-04-21T10:31:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.707+0000 7f44d1043640 1 -- 172.21.15.26:0/2012462687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44cc10a360 con 0x7f44cc1011a0 2024-04-21T10:31:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.708+0000 7f44b3fff640 1 -- 172.21.15.26:0/2012462687 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7f44c4009de0 con 0x7f44cc1011a0 2024-04-21T10:31:04.708 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:04.708 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:30:56.186598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24478,24488]},"id":1}]} 2024-04-21T10:31:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 21 2024-04-21T10:31:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.710+0000 7f44d1043640 1 -- 172.21.15.26:0/2012462687 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f44b40a3ed0 msgr2=0x7f44b40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.710+0000 7f44d1043640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f44b40a3ed0 0x7f44b40a6390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f44c0005fd0 tx=0x7f44c0005ea0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.710+0000 7f44d1043640 1 -- 172.21.15.26:0/2012462687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44cc1011a0 msgr2=0x7f44cc0feb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.710+0000 7f44d1043640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44cc1011a0 0x7f44cc0feb80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f44c400d950 tx=0x7f44c400de10 comp rx=0 tx=0).stop 2024-04-21T10:31:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.710+0000 7f44d1043640 1 -- 172.21.15.26:0/2012462687 shutdown_connections 2024-04-21T10:31:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.710+0000 7f44d1043640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f44b40a3ed0 0x7f44b40a6390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.710+0000 7f44d1043640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44cc105e30 0x7f44cc0ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.710+0000 7f44d1043640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44cc101b60 0x7f44cc0ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.710+0000 7f44d1043640 1 --2- 172.21.15.26:0/2012462687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44cc1011a0 0x7f44cc0feb80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.710+0000 7f44d1043640 1 -- 172.21.15.26:0/2012462687 >> 172.21.15.26:0/2012462687 conn(0x7f44cc077e30 msgr2=0x7f44cc0693e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:04.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.710+0000 7f44d1043640 1 -- 172.21.15.26:0/2012462687 shutdown_connections 2024-04-21T10:31:04.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.710+0000 7f44d1043640 1 -- 172.21.15.26:0/2012462687 wait complete. 2024-04-21T10:31:04.721 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:31:04.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.824+0000 7f1aef78f640 1 -- 172.21.15.26:0/4213004193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae8106f50 msgr2=0x7f1ae8107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.824+0000 7f1aef78f640 1 --2- 172.21.15.26:0/4213004193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae8106f50 0x7f1ae8107330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009900 tx=0x7f1ae402bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:04.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.824+0000 7f1aef78f640 1 -- 172.21.15.26:0/4213004193 shutdown_connections 2024-04-21T10:31:04.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.824+0000 7f1aef78f640 1 --2- 172.21.15.26:0/4213004193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ae8069aa0 0x7f1ae810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.824+0000 7f1aef78f640 1 --2- 172.21.15.26:0/4213004193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae80690e0 0x7f1ae8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.824+0000 7f1aef78f640 1 --2- 172.21.15.26:0/4213004193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae8106f50 0x7f1ae8107330 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.824+0000 7f1aef78f640 1 -- 172.21.15.26:0/4213004193 >> 172.21.15.26:0/4213004193 conn(0x7f1ae8077e30 msgr2=0x7f1ae8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:04.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.824+0000 7f1aef78f640 1 -- 172.21.15.26:0/4213004193 shutdown_connections 2024-04-21T10:31:04.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.824+0000 7f1aef78f640 1 -- 172.21.15.26:0/4213004193 wait complete. 2024-04-21T10:31:04.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.824+0000 7f1aef78f640 1 Processor -- start 2024-04-21T10:31:04.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.824+0000 7f1aef78f640 1 -- start start 2024-04-21T10:31:04.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aef78f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae80690e0 0x7f1ae8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:04.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aef78f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae8069aa0 0x7f1ae8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:04.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aef78f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ae8106f50 0x7f1ae819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:04.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aef78f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8112170 con 0x7f1ae8069aa0 2024-04-21T10:31:04.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aef78f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8111ff0 con 0x7f1ae8106f50 2024-04-21T10:31:04.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aef78f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81122f0 con 0x7f1ae80690e0 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aecd2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae8069aa0 0x7f1ae8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aecd2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae8069aa0 0x7f1ae8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33082/0 (socket says 172.21.15.26:33082) 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aecd2a640 1 -- 172.21.15.26:0/1821833857 learned_addr learned my addr 172.21.15.26:0/1821833857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aedd2c640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ae8106f50 0x7f1ae819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aed52b640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae80690e0 0x7f1ae8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aecd2a640 1 -- 172.21.15.26:0/1821833857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae80690e0 msgr2=0x7f1ae8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aecd2a640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae80690e0 0x7f1ae8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aecd2a640 1 -- 172.21.15.26:0/1821833857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ae8106f50 msgr2=0x7f1ae819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aecd2a640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ae8106f50 0x7f1ae819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aecd2a640 1 -- 172.21.15.26:0/1821833857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae819d6b0 con 0x7f1ae8069aa0 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aedd2c640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ae8106f50 0x7f1ae819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aed52b640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae80690e0 0x7f1ae8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aecd2a640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae8069aa0 0x7f1ae8198c50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800e9f0 tx=0x7f1ad800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1ade7fc640 1 -- 172.21.15.26:0/1821833857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad800cd80 con 0x7f1ae8069aa0 2024-04-21T10:31:04.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aef78f640 1 -- 172.21.15.26:0/1821833857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae819d9a0 con 0x7f1ae8069aa0 2024-04-21T10:31:04.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1ade7fc640 1 -- 172.21.15.26:0/1821833857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad80054d0 con 0x7f1ae8069aa0 2024-04-21T10:31:04.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1ade7fc640 1 -- 172.21.15.26:0/1821833857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8010690 con 0x7f1ae8069aa0 2024-04-21T10:31:04.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.825+0000 7f1aef78f640 1 -- 172.21.15.26:0/1821833857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae81a5230 con 0x7f1ae8069aa0 2024-04-21T10:31:04.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.826+0000 7f1aef78f640 1 -- 172.21.15.26:0/1821833857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae8068de0 con 0x7f1ae8069aa0 2024-04-21T10:31:04.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.828+0000 7f1ade7fc640 1 -- 172.21.15.26:0/1821833857 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f1ad8004e90 con 0x7f1ae8069aa0 2024-04-21T10:31:04.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.828+0000 7f1ade7fc640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1ad40a3ed0 0x7f1ad40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:04.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.828+0000 7f1aed52b640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1ad40a3ed0 0x7f1ad40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:04.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.828+0000 7f1ade7fc640 1 -- 172.21.15.26:0/1821833857 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f1ad80d1010 con 0x7f1ae8069aa0 2024-04-21T10:31:04.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.829+0000 7f1aed52b640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1ad40a3ed0 0x7f1ad40a6390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4036040 tx=0x7f1ae4000bc0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:04.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:04.833+0000 7f1ade7fc640 1 -- 172.21.15.26:0/1821833857 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ad809d480 con 0x7f1ae8069aa0 2024-04-21T10:31:05.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.021+0000 7f1aef78f640 1 -- 172.21.15.26:0/1821833857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ae802d5c0 con 0x7f1ae8069aa0 2024-04-21T10:31:05.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.022+0000 7f1ade7fc640 1 -- 172.21.15.26:0/1821833857 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+23079 (secure 0 0 0) 0x7f1ad809cd80 con 0x7f1ae8069aa0 2024-04-21T10:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:05.022 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":90,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:30:57.801035+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3802401516":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/3125048023":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4245538524":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2797499963":"2024-04-22T10:30:57.800969+0000","172.21.15.26:6801/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:31:05.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.024+0000 7f1aef78f640 1 -- 172.21.15.26:0/1821833857 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1ad40a3ed0 msgr2=0x7f1ad40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.024+0000 7f1aef78f640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1ad40a3ed0 0x7f1ad40a6390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4036040 tx=0x7f1ae4000bc0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.024+0000 7f1aef78f640 1 -- 172.21.15.26:0/1821833857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae8069aa0 msgr2=0x7f1ae8198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.024+0000 7f1aef78f640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae8069aa0 0x7f1ae8198c50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800e9f0 tx=0x7f1ad800edf0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.024+0000 7f1aef78f640 1 -- 172.21.15.26:0/1821833857 shutdown_connections 2024-04-21T10:31:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.024+0000 7f1aef78f640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1ad40a3ed0 0x7f1ad40a6390 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.024+0000 7f1aef78f640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ae8106f50 0x7f1ae819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.024+0000 7f1aef78f640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae8069aa0 0x7f1ae8198c50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.024+0000 7f1aef78f640 1 --2- 172.21.15.26:0/1821833857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae80690e0 0x7f1ae8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.024+0000 7f1aef78f640 1 -- 172.21.15.26:0/1821833857 >> 172.21.15.26:0/1821833857 conn(0x7f1ae8077e30 msgr2=0x7f1ae80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.025+0000 7f1aef78f640 1 -- 172.21.15.26:0/1821833857 shutdown_connections 2024-04-21T10:31:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.025+0000 7f1aef78f640 1 -- 172.21.15.26:0/1821833857 wait complete. 2024-04-21T10:31:05.035 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T10:24:28.176891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T10:28:59.417687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T10:29:00.234808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T10:31:05.036 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T10:31:05.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.143+0000 7f0ceacb3640 1 -- 172.21.15.26:0/2130577343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce410c510 msgr2=0x7f0ce410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.143+0000 7f0ceacb3640 1 --2- 172.21.15.26:0/2130577343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce410c510 0x7f0ce410e9b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009ca0 tx=0x7f0cdc02c230 comp rx=0 tx=0).stop 2024-04-21T10:31:05.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.143+0000 7f0ceacb3640 1 -- 172.21.15.26:0/2130577343 shutdown_connections 2024-04-21T10:31:05.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.143+0000 7f0ceacb3640 1 --2- 172.21.15.26:0/2130577343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce410c510 0x7f0ce410e9b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.143+0000 7f0ceacb3640 1 --2- 172.21.15.26:0/2130577343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce4103af0 0x7f0ce410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.143+0000 7f0ceacb3640 1 --2- 172.21.15.26:0/2130577343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ce41031d0 0x7f0ce41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.143+0000 7f0ceacb3640 1 -- 172.21.15.26:0/2130577343 >> 172.21.15.26:0/2130577343 conn(0x7f0ce4077f50 msgr2=0x7f0ce4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:05.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.143+0000 7f0ceacb3640 1 -- 172.21.15.26:0/2130577343 shutdown_connections 2024-04-21T10:31:05.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.143+0000 7f0ceacb3640 1 -- 172.21.15.26:0/2130577343 wait complete. 2024-04-21T10:31:05.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ceacb3640 1 Processor -- start 2024-04-21T10:31:05.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ceacb3640 1 -- start start 2024-04-21T10:31:05.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ceacb3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ce41031d0 0x7f0ce41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:05.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ceacb3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce4103af0 0x7f0ce4198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:05.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ceacb3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce410c510 0x7f0ce419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:05.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ceacb3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce41121a0 con 0x7f0ce4103af0 2024-04-21T10:31:05.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ceacb3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4112020 con 0x7f0ce41031d0 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ceacb3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4112320 con 0x7f0ce410c510 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce4103af0 0x7f0ce4198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ce3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce4103af0 0x7f0ce4198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33098/0 (socket says 172.21.15.26:33098) 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ce3fff640 1 -- 172.21.15.26:0/874657805 learned_addr learned my addr 172.21.15.26:0/874657805 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ce9250640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce410c510 0x7f0ce419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ce8a4f640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ce41031d0 0x7f0ce41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ce3fff640 1 -- 172.21.15.26:0/874657805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce410c510 msgr2=0x7f0ce419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ce3fff640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce410c510 0x7f0ce419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ce3fff640 1 -- 172.21.15.26:0/874657805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ce41031d0 msgr2=0x7f0ce41986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ce3fff640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ce41031d0 0x7f0ce41986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.144+0000 7f0ce3fff640 1 -- 172.21.15.26:0/874657805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce419d6d0 con 0x7f0ce4103af0 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.145+0000 7f0ce3fff640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce4103af0 0x7f0ce4198c20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000e9f0 tx=0x7f0cd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.145+0000 7f0ce1ffb640 1 -- 172.21.15.26:0/874657805 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd000cd80 con 0x7f0ce4103af0 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.145+0000 7f0ce1ffb640 1 -- 172.21.15.26:0/874657805 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cd00054d0 con 0x7f0ce4103af0 2024-04-21T10:31:05.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.145+0000 7f0ce1ffb640 1 -- 172.21.15.26:0/874657805 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0010690 con 0x7f0ce4103af0 2024-04-21T10:31:05.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.145+0000 7f0ceacb3640 1 -- 172.21.15.26:0/874657805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce419d9c0 con 0x7f0ce4103af0 2024-04-21T10:31:05.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.145+0000 7f0ceacb3640 1 -- 172.21.15.26:0/874657805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce41a5250 con 0x7f0ce4103af0 2024-04-21T10:31:05.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.146+0000 7f0ceacb3640 1 -- 172.21.15.26:0/874657805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce4068d70 con 0x7f0ce4103af0 2024-04-21T10:31:05.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.147+0000 7f0ce1ffb640 1 -- 172.21.15.26:0/874657805 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f0cd0004e90 con 0x7f0ce4103af0 2024-04-21T10:31:05.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.147+0000 7f0ce1ffb640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0cd40a3f50 0x7f0cd40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:05.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.148+0000 7f0ce1ffb640 1 -- 172.21.15.26:0/874657805 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f0cd00d0df0 con 0x7f0ce4103af0 2024-04-21T10:31:05.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.148+0000 7f0ce8a4f640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0cd40a3f50 0x7f0cd40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:05.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.148+0000 7f0ce8a4f640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0cd40a3f50 0x7f0cd40a6410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009bd0 tx=0x7f0cd8005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:05.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.152+0000 7f0ce1ffb640 1 -- 172.21.15.26:0/874657805 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cd009d260 con 0x7f0ce4103af0 2024-04-21T10:31:05.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.339+0000 7f0ceacb3640 1 -- 172.21.15.26:0/874657805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ce41076f0 con 0x7f0ce4103af0 2024-04-21T10:31:05.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.340+0000 7f0ce1ffb640 1 -- 172.21.15.26:0/874657805 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f0cd009cb60 con 0x7f0ce4103af0 2024-04-21T10:31:05.339 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-21T10:31:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.342+0000 7f0ceacb3640 1 -- 172.21.15.26:0/874657805 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0cd40a3f50 msgr2=0x7f0cd40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.342+0000 7f0ceacb3640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0cd40a3f50 0x7f0cd40a6410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009bd0 tx=0x7f0cd8005f90 comp rx=0 tx=0).stop 2024-04-21T10:31:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.342+0000 7f0ceacb3640 1 -- 172.21.15.26:0/874657805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce4103af0 msgr2=0x7f0ce4198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.342+0000 7f0ceacb3640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce4103af0 0x7f0ce4198c20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000e9f0 tx=0x7f0cd000edf0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.342+0000 7f0ceacb3640 1 -- 172.21.15.26:0/874657805 shutdown_connections 2024-04-21T10:31:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.342+0000 7f0ceacb3640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0cd40a3f50 0x7f0cd40a6410 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.342+0000 7f0ceacb3640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ce410c510 0x7f0ce419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.342+0000 7f0ceacb3640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce4103af0 0x7f0ce4198c20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.342+0000 7f0ceacb3640 1 --2- 172.21.15.26:0/874657805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ce41031d0 0x7f0ce41986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.342+0000 7f0ceacb3640 1 -- 172.21.15.26:0/874657805 >> 172.21.15.26:0/874657805 conn(0x7f0ce4077f50 msgr2=0x7f0ce40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:05.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.342+0000 7f0ceacb3640 1 -- 172.21.15.26:0/874657805 shutdown_connections 2024-04-21T10:31:05.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.342+0000 7f0ceacb3640 1 -- 172.21.15.26:0/874657805 wait complete. 2024-04-21T10:31:05.353 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T10:31:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.475+0000 7f0f68323640 1 -- 172.21.15.26:0/3296057746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f60074220 msgr2=0x7f0f60074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.475+0000 7f0f68323640 1 --2- 172.21.15.26:0/3296057746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f60074220 0x7f0f60074600 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009900 tx=0x7f0f5c02bcd0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.475+0000 7f0f68323640 1 -- 172.21.15.26:0/3296057746 shutdown_connections 2024-04-21T10:31:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.475+0000 7f0f68323640 1 --2- 172.21.15.26:0/3296057746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f6010e830 0x7f0f60110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.475+0000 7f0f68323640 1 --2- 172.21.15.26:0/3296057746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f60074b40 0x7f0f6010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.475+0000 7f0f68323640 1 --2- 172.21.15.26:0/3296057746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f60074220 0x7f0f60074600 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.475+0000 7f0f68323640 1 -- 172.21.15.26:0/3296057746 >> 172.21.15.26:0/3296057746 conn(0x7f0f600793a0 msgr2=0x7f0f600797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.475+0000 7f0f68323640 1 -- 172.21.15.26:0/3296057746 shutdown_connections 2024-04-21T10:31:05.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.475+0000 7f0f68323640 1 -- 172.21.15.26:0/3296057746 wait complete. 2024-04-21T10:31:05.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f68323640 1 Processor -- start 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f68323640 1 -- start start 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f68323640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f60074220 0x7f0f601a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f68323640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f60074b40 0x7f0f601a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f68323640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f6010e830 0x7f0f6019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f68323640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0f60114440 con 0x7f0f60074b40 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f68323640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f601142c0 con 0x7f0f60074220 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f68323640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0f601145c0 con 0x7f0f6010e830 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f658be640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f60074b40 0x7f0f601a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f658be640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f60074b40 0x7f0f601a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33118/0 (socket says 172.21.15.26:33118) 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f658be640 1 -- 172.21.15.26:0/155694986 learned_addr learned my addr 172.21.15.26:0/155694986 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f660bf640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f60074220 0x7f0f601a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.476+0000 7f0f668c0640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f6010e830 0x7f0f6019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.477+0000 7f0f658be640 1 -- 172.21.15.26:0/155694986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f6010e830 msgr2=0x7f0f6019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.477+0000 7f0f658be640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f6010e830 0x7f0f6019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.477+0000 7f0f658be640 1 -- 172.21.15.26:0/155694986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f60074220 msgr2=0x7f0f601a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.477+0000 7f0f658be640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f60074220 0x7f0f601a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.477+0000 7f0f658be640 1 -- 172.21.15.26:0/155694986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6019af90 con 0x7f0f60074b40 2024-04-21T10:31:05.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.477+0000 7f0f660bf640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f60074220 0x7f0f601a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:31:05.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.477+0000 7f0f658be640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f60074b40 0x7f0f601a0b90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000e9f0 tx=0x7f0f5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:05.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.477+0000 7f0f577fe640 1 -- 172.21.15.26:0/155694986 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5000cd80 con 0x7f0f60074b40 2024-04-21T10:31:05.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.477+0000 7f0f577fe640 1 -- 172.21.15.26:0/155694986 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f500108e0 con 0x7f0f60074b40 2024-04-21T10:31:05.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.477+0000 7f0f68323640 1 -- 172.21.15.26:0/155694986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6019b280 con 0x7f0f60074b40 2024-04-21T10:31:05.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.477+0000 7f0f577fe640 1 -- 172.21.15.26:0/155694986 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50018980 con 0x7f0f60074b40 2024-04-21T10:31:05.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.477+0000 7f0f68323640 1 -- 172.21.15.26:0/155694986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f601a74a0 con 0x7f0f60074b40 2024-04-21T10:31:05.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.478+0000 7f0f68323640 1 -- 172.21.15.26:0/155694986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f60075960 con 0x7f0f60074b40 2024-04-21T10:31:05.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.479+0000 7f0f577fe640 1 -- 172.21.15.26:0/155694986 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f0f50010430 con 0x7f0f60074b40 2024-04-21T10:31:05.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.480+0000 7f0f577fe640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0f4c0a3f50 0x7f0f4c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:05.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.480+0000 7f0f660bf640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0f4c0a3f50 0x7f0f4c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:05.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.480+0000 7f0f577fe640 1 -- 172.21.15.26:0/155694986 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f0f500d1390 con 0x7f0f60074b40 2024-04-21T10:31:05.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.481+0000 7f0f660bf640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0f4c0a3f50 0x7f0f4c0a6410 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c006010 tx=0x7f0f5c005a20 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:05.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.484+0000 7f0f577fe640 1 -- 172.21.15.26:0/155694986 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f5009d800 con 0x7f0f60074b40 2024-04-21T10:31:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.672+0000 7f0f68323640 1 -- 172.21.15.26:0/155694986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0f601099b0 con 0x7f0f60074b40 2024-04-21T10:31:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.672+0000 7f0f577fe640 1 -- 172.21.15.26:0/155694986 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f0f5009d100 con 0x7f0f60074b40 2024-04-21T10:31:05.672 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-21T10:31:05.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.674+0000 7f0f68323640 1 -- 172.21.15.26:0/155694986 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0f4c0a3f50 msgr2=0x7f0f4c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.674+0000 7f0f68323640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0f4c0a3f50 0x7f0f4c0a6410 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c006010 tx=0x7f0f5c005a20 comp rx=0 tx=0).stop 2024-04-21T10:31:05.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.674+0000 7f0f68323640 1 -- 172.21.15.26:0/155694986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f60074b40 msgr2=0x7f0f601a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.674+0000 7f0f68323640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f60074b40 0x7f0f601a0b90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000e9f0 tx=0x7f0f5000eeb0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.674+0000 7f0f68323640 1 -- 172.21.15.26:0/155694986 shutdown_connections 2024-04-21T10:31:05.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.674+0000 7f0f68323640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0f4c0a3f50 0x7f0f4c0a6410 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.674+0000 7f0f68323640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f6010e830 0x7f0f6019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.674+0000 7f0f68323640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f60074b40 0x7f0f601a0b90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.674+0000 7f0f68323640 1 --2- 172.21.15.26:0/155694986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f60074220 0x7f0f601a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.674+0000 7f0f68323640 1 -- 172.21.15.26:0/155694986 >> 172.21.15.26:0/155694986 conn(0x7f0f600793a0 msgr2=0x7f0f600feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.675+0000 7f0f68323640 1 -- 172.21.15.26:0/155694986 shutdown_connections 2024-04-21T10:31:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.675+0000 7f0f68323640 1 -- 172.21.15.26:0/155694986 wait complete. 2024-04-21T10:31:05.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:05 smithi026 ceph-mon[27965]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:31:05.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:05 smithi026 ceph-mon[27965]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-21T10:31:05.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:05 smithi026 ceph-mon[27965]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:31:05.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:05 smithi026 ceph-mon[27965]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:31:05.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:05 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2012462687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:05.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:05 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1821833857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:31:05.686 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T10:31:05.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:05 smithi169 ceph-mon[31469]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:31:05.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:05 smithi169 ceph-mon[31469]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-21T10:31:05.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:05 smithi169 ceph-mon[31469]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:31:05.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:05 smithi169 ceph-mon[31469]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:31:05.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:05 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2012462687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:05.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:05 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1821833857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:31:05.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:05 smithi190 ceph-mon[31423]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:31:05.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:05 smithi190 ceph-mon[31423]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-21T10:31:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:05 smithi190 ceph-mon[31423]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:31:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:05 smithi190 ceph-mon[31423]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T10:31:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:05 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2012462687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:05 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1821833857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:31:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.793+0000 7fdbc7226640 1 -- 172.21.15.26:0/1261233621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0101090 msgr2=0x7fdbc0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.793+0000 7fdbc7226640 1 --2- 172.21.15.26:0/1261233621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0101090 0x7fdbc0101470 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009ca0 tx=0x7fdbb802c0f0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.793+0000 7fdbc7226640 1 -- 172.21.15.26:0/1261233621 shutdown_connections 2024-04-21T10:31:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.793+0000 7fdbc7226640 1 --2- 172.21.15.26:0/1261233621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbc0106160 0x7fdbc010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.793+0000 7fdbc7226640 1 --2- 172.21.15.26:0/1261233621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbc0101a40 0x7fdbc0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.793+0000 7fdbc7226640 1 --2- 172.21.15.26:0/1261233621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0101090 0x7fdbc0101470 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.793+0000 7fdbc7226640 1 -- 172.21.15.26:0/1261233621 >> 172.21.15.26:0/1261233621 conn(0x7fdbc0077f50 msgr2=0x7fdbc0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.793+0000 7fdbc7226640 1 -- 172.21.15.26:0/1261233621 shutdown_connections 2024-04-21T10:31:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.793+0000 7fdbc7226640 1 -- 172.21.15.26:0/1261233621 wait complete. 2024-04-21T10:31:05.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbc7226640 1 Processor -- start 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbc7226640 1 -- start start 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbc7226640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbc0101090 0x7fdbc0197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbc7226640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0106160 0x7fdbc0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbc7226640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbc019bf10 0x7fdbc019c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbc7226640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc01110d0 con 0x7fdbc0106160 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbc7226640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc0110f50 con 0x7fdbc0101090 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbc7226640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc0111250 con 0x7fdbc019bf10 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbbffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0106160 0x7fdbc0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbbffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0106160 0x7fdbc0197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33144/0 (socket says 172.21.15.26:33144) 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbbffff640 1 -- 172.21.15.26:0/627274522 learned_addr learned my addr 172.21.15.26:0/627274522 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbc57c3640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbc019bf10 0x7fdbc019c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbc4fc2640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbc0101090 0x7fdbc0197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:05.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbbffff640 1 -- 172.21.15.26:0/627274522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbc019bf10 msgr2=0x7fdbc019c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbbffff640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbc019bf10 0x7fdbc019c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbbffff640 1 -- 172.21.15.26:0/627274522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbc0101090 msgr2=0x7fdbc0197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbbffff640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbc0101090 0x7fdbc0197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbbffff640 1 -- 172.21.15.26:0/627274522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc019ca00 con 0x7fdbc0106160 2024-04-21T10:31:05.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbc4fc2640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbc0101090 0x7fdbc0197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:31:05.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.794+0000 7fdbbffff640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0106160 0x7fdbc0197b90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400e9c0 tx=0x7fdbb400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:05.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.795+0000 7fdbbdffb640 1 -- 172.21.15.26:0/627274522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb400cd80 con 0x7fdbc0106160 2024-04-21T10:31:05.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.795+0000 7fdbbdffb640 1 -- 172.21.15.26:0/627274522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbb40054d0 con 0x7fdbc0106160 2024-04-21T10:31:05.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.795+0000 7fdbc7226640 1 -- 172.21.15.26:0/627274522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc019ccf0 con 0x7fdbc0106160 2024-04-21T10:31:05.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.795+0000 7fdbbdffb640 1 -- 172.21.15.26:0/627274522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4010430 con 0x7fdbc0106160 2024-04-21T10:31:05.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.795+0000 7fdbc7226640 1 -- 172.21.15.26:0/627274522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc0069400 con 0x7fdbc0106160 2024-04-21T10:31:05.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.796+0000 7fdbc7226640 1 -- 172.21.15.26:0/627274522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc0068d70 con 0x7fdbc0106160 2024-04-21T10:31:05.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.797+0000 7fdbbdffb640 1 -- 172.21.15.26:0/627274522 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fdbb4004fd0 con 0x7fdbc0106160 2024-04-21T10:31:05.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.797+0000 7fdbbdffb640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fdba80a3ed0 0x7fdba80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:05.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.798+0000 7fdbbdffb640 1 -- 172.21.15.26:0/627274522 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fdbb40d0f80 con 0x7fdbc0106160 2024-04-21T10:31:05.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.798+0000 7fdbc4fc2640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fdba80a3ed0 0x7fdba80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:05.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.798+0000 7fdbc4fc2640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fdba80a3ed0 0x7fdba80a6390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8000f80 tx=0x7fdbb8005c50 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:05.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.802+0000 7fdbbdffb640 1 -- 172.21.15.26:0/627274522 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdbb409d3f0 con 0x7fdbc0106160 2024-04-21T10:31:05.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.988+0000 7fdbc7226640 1 -- 172.21.15.26:0/627274522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdbc0069f50 con 0x7fdbc0106160 2024-04-21T10:31:05.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.988+0000 7fdbbdffb640 1 -- 172.21.15.26:0/627274522 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fdbb409ccf0 con 0x7fdbc0106160 2024-04-21T10:31:05.988 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-21T10:31:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.991+0000 7fdbc7226640 1 -- 172.21.15.26:0/627274522 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fdba80a3ed0 msgr2=0x7fdba80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.991+0000 7fdbc7226640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fdba80a3ed0 0x7fdba80a6390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8000f80 tx=0x7fdbb8005c50 comp rx=0 tx=0).stop 2024-04-21T10:31:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.991+0000 7fdbc7226640 1 -- 172.21.15.26:0/627274522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0106160 msgr2=0x7fdbc0197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.991+0000 7fdbc7226640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0106160 0x7fdbc0197b90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400e9c0 tx=0x7fdbb400ee80 comp rx=0 tx=0).stop 2024-04-21T10:31:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.991+0000 7fdbc7226640 1 -- 172.21.15.26:0/627274522 shutdown_connections 2024-04-21T10:31:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.991+0000 7fdbc7226640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fdba80a3ed0 0x7fdba80a6390 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.991+0000 7fdbc7226640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbc019bf10 0x7fdbc019c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.991+0000 7fdbc7226640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbc0106160 0x7fdbc0197b90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.991+0000 7fdbc7226640 1 --2- 172.21.15.26:0/627274522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbc0101090 0x7fdbc0197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.991+0000 7fdbc7226640 1 -- 172.21.15.26:0/627274522 >> 172.21.15.26:0/627274522 conn(0x7fdbc0077f50 msgr2=0x7fdbc00ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.991+0000 7fdbc7226640 1 -- 172.21.15.26:0/627274522 shutdown_connections 2024-04-21T10:31:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:05.991+0000 7fdbc7226640 1 -- 172.21.15.26:0/627274522 wait complete. 2024-04-21T10:31:06.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:31:06.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.104+0000 7fbe3cff4640 1 -- 172.21.15.26:0/1227304927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe38069aa0 msgr2=0x7fbe3810ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.104+0000 7fbe3cff4640 1 --2- 172.21.15.26:0/1227304927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe38069aa0 0x7fbe3810ede0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009ca0 tx=0x7fbe3002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.104+0000 7fbe3cff4640 1 -- 172.21.15.26:0/1227304927 shutdown_connections 2024-04-21T10:31:06.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.104+0000 7fbe3cff4640 1 --2- 172.21.15.26:0/1227304927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe38069aa0 0x7fbe3810ede0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.104+0000 7fbe3cff4640 1 --2- 172.21.15.26:0/1227304927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe380690e0 0x7fbe38069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.104+0000 7fbe3cff4640 1 --2- 172.21.15.26:0/1227304927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe38106f50 0x7fbe38107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.104+0000 7fbe3cff4640 1 -- 172.21.15.26:0/1227304927 >> 172.21.15.26:0/1227304927 conn(0x7fbe38077e30 msgr2=0x7fbe38078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.104+0000 7fbe3cff4640 1 -- 172.21.15.26:0/1227304927 shutdown_connections 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.104+0000 7fbe3cff4640 1 -- 172.21.15.26:0/1227304927 wait complete. 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.105+0000 7fbe3cff4640 1 Processor -- start 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.105+0000 7fbe3cff4640 1 -- start start 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.105+0000 7fbe3cff4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380690e0 0x7fbe38101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.105+0000 7fbe3cff4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe38069aa0 0x7fbe381017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.105+0000 7fbe3cff4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe38106f50 0x7fbe38101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.105+0000 7fbe3cff4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38112150 con 0x7fbe380690e0 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.105+0000 7fbe3cff4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38111fd0 con 0x7fbe38069aa0 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.105+0000 7fbe3cff4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbe381122d0 con 0x7fbe38106f50 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.105+0000 7fbe3659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380690e0 0x7fbe38101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.105+0000 7fbe3659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380690e0 0x7fbe38101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33158/0 (socket says 172.21.15.26:33158) 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.105+0000 7fbe3659c640 1 -- 172.21.15.26:0/89724914 learned_addr learned my addr 172.21.15.26:0/89724914 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.105+0000 7fbe35d9b640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe38069aa0 0x7fbe381017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.106+0000 7fbe36d9d640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe38106f50 0x7fbe38101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:06.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.106+0000 7fbe3659c640 1 -- 172.21.15.26:0/89724914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe38106f50 msgr2=0x7fbe38101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.106+0000 7fbe3659c640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe38106f50 0x7fbe38101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.106+0000 7fbe3659c640 1 -- 172.21.15.26:0/89724914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe38069aa0 msgr2=0x7fbe381017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.106+0000 7fbe3659c640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe38069aa0 0x7fbe381017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.106+0000 7fbe3659c640 1 -- 172.21.15.26:0/89724914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe381a6ab0 con 0x7fbe380690e0 2024-04-21T10:31:06.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.106+0000 7fbe3659c640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380690e0 0x7fbe38101270 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000e9f0 tx=0x7fbe2000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:06.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.106+0000 7fbe2f7fe640 1 -- 172.21.15.26:0/89724914 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000cd80 con 0x7fbe380690e0 2024-04-21T10:31:06.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.106+0000 7fbe3cff4640 1 -- 172.21.15.26:0/89724914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe381a6da0 con 0x7fbe380690e0 2024-04-21T10:31:06.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.106+0000 7fbe2f7fe640 1 -- 172.21.15.26:0/89724914 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe200108e0 con 0x7fbe380690e0 2024-04-21T10:31:06.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.106+0000 7fbe2f7fe640 1 -- 172.21.15.26:0/89724914 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20018980 con 0x7fbe380690e0 2024-04-21T10:31:06.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.106+0000 7fbe3cff4640 1 -- 172.21.15.26:0/89724914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe381a7270 con 0x7fbe380690e0 2024-04-21T10:31:06.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.107+0000 7fbe3cff4640 1 -- 172.21.15.26:0/89724914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe38068de0 con 0x7fbe380690e0 2024-04-21T10:31:06.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.108+0000 7fbe2f7fe640 1 -- 172.21.15.26:0/89724914 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fbe20010430 con 0x7fbe380690e0 2024-04-21T10:31:06.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.109+0000 7fbe2f7fe640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fbe240a3f00 0x7fbe240a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:06.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.109+0000 7fbe35d9b640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fbe240a3f00 0x7fbe240a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:06.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.109+0000 7fbe2f7fe640 1 -- 172.21.15.26:0/89724914 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fbe200d0db0 con 0x7fbe380690e0 2024-04-21T10:31:06.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.109+0000 7fbe35d9b640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fbe240a3f00 0x7fbe240a63c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbe38102cb0 tx=0x7fbe2800b390 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.113+0000 7fbe2f7fe640 1 -- 172.21.15.26:0/89724914 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe2009d220 con 0x7fbe380690e0 2024-04-21T10:31:06.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.362+0000 7fbe3cff4640 1 -- 172.21.15.26:0/89724914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe38109f10 con 0x7fbe380690e0 2024-04-21T10:31:06.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.363+0000 7fbe2f7fe640 1 -- 172.21.15.26:0/89724914 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7fbe2009cb20 con 0x7fbe380690e0 2024-04-21T10:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:06.362 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:30:56.186598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24478,24488]},"id":1}]} 2024-04-21T10:31:06.363 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 21 2024-04-21T10:31:06.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.365+0000 7fbe3cff4640 1 -- 172.21.15.26:0/89724914 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fbe240a3f00 msgr2=0x7fbe240a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.365+0000 7fbe3cff4640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fbe240a3f00 0x7fbe240a63c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbe38102cb0 tx=0x7fbe2800b390 comp rx=0 tx=0).stop 2024-04-21T10:31:06.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.365+0000 7fbe3cff4640 1 -- 172.21.15.26:0/89724914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380690e0 msgr2=0x7fbe38101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.365+0000 7fbe3cff4640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380690e0 0x7fbe38101270 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000e9f0 tx=0x7fbe2000eeb0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.365+0000 7fbe3cff4640 1 -- 172.21.15.26:0/89724914 shutdown_connections 2024-04-21T10:31:06.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.365+0000 7fbe3cff4640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fbe240a3f00 0x7fbe240a63c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.365+0000 7fbe3cff4640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe38106f50 0x7fbe38101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.365+0000 7fbe3cff4640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe38069aa0 0x7fbe381017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.365+0000 7fbe3cff4640 1 --2- 172.21.15.26:0/89724914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380690e0 0x7fbe38101270 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.365+0000 7fbe3cff4640 1 -- 172.21.15.26:0/89724914 >> 172.21.15.26:0/89724914 conn(0x7fbe38077e30 msgr2=0x7fbe380fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:06.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.365+0000 7fbe3cff4640 1 -- 172.21.15.26:0/89724914 shutdown_connections 2024-04-21T10:31:06.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.365+0000 7fbe3cff4640 1 -- 172.21.15.26:0/89724914 wait complete. 2024-04-21T10:31:06.376 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:31:06.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.478+0000 7f56af74b640 1 -- 172.21.15.26:0/2639594913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a810c5a0 msgr2=0x7f56a810e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.478+0000 7f56af74b640 1 --2- 172.21.15.26:0/2639594913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a810c5a0 0x7f56a810e9d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009ca0 tx=0x7f56a002c180 comp rx=0 tx=0).stop 2024-04-21T10:31:06.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.478+0000 7f56af74b640 1 -- 172.21.15.26:0/2639594913 shutdown_connections 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.478+0000 7f56af74b640 1 --2- 172.21.15.26:0/2639594913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a810c5a0 0x7f56a810e9d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.478+0000 7f56af74b640 1 --2- 172.21.15.26:0/2639594913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56a8103af0 0x7f56a810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.478+0000 7f56af74b640 1 --2- 172.21.15.26:0/2639594913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a81031d0 0x7f56a81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.478+0000 7f56af74b640 1 -- 172.21.15.26:0/2639594913 >> 172.21.15.26:0/2639594913 conn(0x7f56a80771d0 msgr2=0x7f56a80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.478+0000 7f56af74b640 1 -- 172.21.15.26:0/2639594913 shutdown_connections 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.478+0000 7f56af74b640 1 -- 172.21.15.26:0/2639594913 wait complete. 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56af74b640 1 Processor -- start 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56af74b640 1 -- start start 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56af74b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a81031d0 0x7f56a8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56af74b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56a8103af0 0x7f56a8198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56af74b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a810c5a0 0x7f56a819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56af74b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56a81121e0 con 0x7f56a81031d0 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56af74b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56a8112060 con 0x7f56a8103af0 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56af74b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f56a8112360 con 0x7f56a810c5a0 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56ad4e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a81031d0 0x7f56a8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56ad4e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a81031d0 0x7f56a8198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33186/0 (socket says 172.21.15.26:33186) 2024-04-21T10:31:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56ad4e7640 1 -- 172.21.15.26:0/837684328 learned_addr learned my addr 172.21.15.26:0/837684328 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56ad4e7640 1 -- 172.21.15.26:0/837684328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a810c5a0 msgr2=0x7f56a819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56ad4e7640 1 --2- 172.21.15.26:0/837684328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a810c5a0 0x7f56a819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56ad4e7640 1 -- 172.21.15.26:0/837684328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56a8103af0 msgr2=0x7f56a8198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56ad4e7640 1 --2- 172.21.15.26:0/837684328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56a8103af0 0x7f56a8198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56ad4e7640 1 -- 172.21.15.26:0/837684328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a819d6f0 con 0x7f56a81031d0 2024-04-21T10:31:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.479+0000 7f56ad4e7640 1 --2- 172.21.15.26:0/837684328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a81031d0 0x7f56a8198700 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f56a400d770 tx=0x7f56a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.480+0000 7f569e7fc640 1 -- 172.21.15.26:0/837684328 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4017070 con 0x7f56a81031d0 2024-04-21T10:31:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.480+0000 7f569e7fc640 1 -- 172.21.15.26:0/837684328 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a40054d0 con 0x7f56a81031d0 2024-04-21T10:31:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.480+0000 7f56af74b640 1 -- 172.21.15.26:0/837684328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a819d9e0 con 0x7f56a81031d0 2024-04-21T10:31:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.480+0000 7f569e7fc640 1 -- 172.21.15.26:0/837684328 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4004c30 con 0x7f56a81031d0 2024-04-21T10:31:06.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.480+0000 7f56af74b640 1 -- 172.21.15.26:0/837684328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a81a5270 con 0x7f56a81031d0 2024-04-21T10:31:06.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.481+0000 7f56af74b640 1 -- 172.21.15.26:0/837684328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a8069230 con 0x7f56a81031d0 2024-04-21T10:31:06.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.482+0000 7f569e7fc640 1 -- 172.21.15.26:0/837684328 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f56a4004dd0 con 0x7f56a81031d0 2024-04-21T10:31:06.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.482+0000 7f569e7fc640 1 --2- 172.21.15.26:0/837684328 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f56940a3ed0 0x7f56940a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:06.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.483+0000 7f56acce6640 1 --2- 172.21.15.26:0/837684328 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f56940a3ed0 0x7f56940a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:06.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.483+0000 7f569e7fc640 1 -- 172.21.15.26:0/837684328 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f56a40d0b60 con 0x7f56a81031d0 2024-04-21T10:31:06.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.483+0000 7f56acce6640 1 --2- 172.21.15.26:0/837684328 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f56940a3ed0 0x7f56940a6390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5698005d80 tx=0x7f5698005c90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:06.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.486+0000 7f569e7fc640 1 -- 172.21.15.26:0/837684328 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56a409cef0 con 0x7f56a81031d0 2024-04-21T10:31:06.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.675+0000 7f56af74b640 1 -- 172.21.15.26:0/837684328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56a802d5c0 con 0x7f56a81031d0 2024-04-21T10:31:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.677+0000 7f569e7fc640 1 -- 172.21.15.26:0/837684328 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+23079 (secure 0 0 0) 0x7f56a409c7f0 con 0x7f56a81031d0 2024-04-21T10:31:06.676 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:06.677 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":90,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:30:57.801035+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3802401516":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/3125048023":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4245538524":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2797499963":"2024-04-22T10:30:57.800969+0000","172.21.15.26:6801/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:31:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.679+0000 7f56af74b640 1 -- 172.21.15.26:0/837684328 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f56940a3ed0 msgr2=0x7f56940a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.679+0000 7f56af74b640 1 --2- 172.21.15.26:0/837684328 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f56940a3ed0 0x7f56940a6390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5698005d80 tx=0x7f5698005c90 comp rx=0 tx=0).stop 2024-04-21T10:31:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.679+0000 7f56af74b640 1 -- 172.21.15.26:0/837684328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a81031d0 msgr2=0x7f56a8198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.679+0000 7f56af74b640 1 --2- 172.21.15.26:0/837684328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a81031d0 0x7f56a8198700 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f56a400d770 tx=0x7f56a400dc30 comp rx=0 tx=0).stop 2024-04-21T10:31:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.679+0000 7f56af74b640 1 -- 172.21.15.26:0/837684328 shutdown_connections 2024-04-21T10:31:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.679+0000 7f56af74b640 1 --2- 172.21.15.26:0/837684328 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f56940a3ed0 0x7f56940a6390 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.679+0000 7f56af74b640 1 --2- 172.21.15.26:0/837684328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56a810c5a0 0x7f56a819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.679+0000 7f56af74b640 1 --2- 172.21.15.26:0/837684328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56a8103af0 0x7f56a8198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.679+0000 7f56af74b640 1 --2- 172.21.15.26:0/837684328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a81031d0 0x7f56a8198700 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.679+0000 7f56af74b640 1 -- 172.21.15.26:0/837684328 >> 172.21.15.26:0/837684328 conn(0x7f56a80771d0 msgr2=0x7f56a80fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.679+0000 7f56af74b640 1 -- 172.21.15.26:0/837684328 shutdown_connections 2024-04-21T10:31:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.679+0000 7f56af74b640 1 -- 172.21.15.26:0/837684328 wait complete. 2024-04-21T10:31:06.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:06 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/874657805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:31:06.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:06 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/155694986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:31:06.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:06 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/627274522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:31:06.690 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 800522 2024-04-21T10:31:06.690 DEBUG:tasks.quiescer:Yielding 2024-04-21T10:31:06.690 INFO:teuthology.run_tasks:Running task workunit... 2024-04-21T10:31:06.703 INFO:tasks.workunit:Pulling workunits from ref b8ddeda347de6c45510a6df67440a1d298c37030 2024-04-21T10:31:06.703 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-21T10:31:06.704 INFO:tasks.workunit:timeout=3h 2024-04-21T10:31:06.704 INFO:tasks.workunit:cleanup=True 2024-04-21T10:31:06.704 DEBUG:teuthology.orchestra.run.smithi026:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-21T10:31:06.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:31:06.705 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:06 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/874657805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:31:06.705 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:06 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/155694986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:31:06.705 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:06 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/627274522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:31:06.720 INFO:teuthology.orchestra.run.smithi026.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-21T10:31:06.720 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-21T10:31:06.720 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-21T10:31:06.720 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T10:31:06.720 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-21T10:31:06.720 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-21 10:29:07.298117258 +0000 2024-04-21T10:31:06.720 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-21 10:29:07.298117258 +0000 2024-04-21T10:31:06.720 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-21 10:30:55.490319665 +0000 2024-04-21T10:31:06.720 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-21 10:29:07.298117258 +0000 2024-04-21T10:31:06.721 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-21T10:31:06.721 DEBUG:teuthology.orchestra.run.smithi026:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-21T10:31:06.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:06 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/874657805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:31:06.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:06 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/155694986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:31:06.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:06 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/627274522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:31:06.793 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b8ddeda347de6c45510a6df67440a1d298c37030 2024-04-21T10:31:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.808+0000 7fbcd45c3640 1 -- 172.21.15.26:0/1855492406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1031d0 msgr2=0x7fbccc1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.808+0000 7fbcd45c3640 1 --2- 172.21.15.26:0/1855492406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1031d0 0x7fbccc1035b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009900 tx=0x7fbcc802bcd0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.808+0000 7fbcd45c3640 1 -- 172.21.15.26:0/1855492406 shutdown_connections 2024-04-21T10:31:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.808+0000 7fbcd45c3640 1 --2- 172.21.15.26:0/1855492406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbccc10c5a0 0x7fbccc10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.808+0000 7fbcd45c3640 1 --2- 172.21.15.26:0/1855492406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc103af0 0x7fbccc10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.808+0000 7fbcd45c3640 1 --2- 172.21.15.26:0/1855492406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1031d0 0x7fbccc1035b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.808+0000 7fbcd45c3640 1 -- 172.21.15.26:0/1855492406 >> 172.21.15.26:0/1855492406 conn(0x7fbccc0771d0 msgr2=0x7fbccc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.808+0000 7fbcd45c3640 1 -- 172.21.15.26:0/1855492406 shutdown_connections 2024-04-21T10:31:06.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.808+0000 7fbcd45c3640 1 -- 172.21.15.26:0/1855492406 wait complete. 2024-04-21T10:31:06.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd45c3640 1 Processor -- start 2024-04-21T10:31:06.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd45c3640 1 -- start start 2024-04-21T10:31:06.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd45c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1031d0 0x7fbccc100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:06.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd45c3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbccc103af0 0x7fbccc101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:06.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd45c3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc10c5a0 0x7fbccc1018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:06.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd45c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1121d0 con 0x7fbccc1031d0 2024-04-21T10:31:06.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd45c3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc112050 con 0x7fbccc10c5a0 2024-04-21T10:31:06.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd45c3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc112350 con 0x7fbccc103af0 2024-04-21T10:31:06.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd235f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1031d0 0x7fbccc100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd235f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1031d0 0x7fbccc100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33206/0 (socket says 172.21.15.26:33206) 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd235f640 1 -- 172.21.15.26:0/4197061070 learned_addr learned my addr 172.21.15.26:0/4197061070 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd2b60640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc10c5a0 0x7fbccc1018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd1b5e640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbccc103af0 0x7fbccc101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.809+0000 7fbcd235f640 1 -- 172.21.15.26:0/4197061070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbccc103af0 msgr2=0x7fbccc101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.810+0000 7fbcd235f640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbccc103af0 0x7fbccc101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.810+0000 7fbcd235f640 1 -- 172.21.15.26:0/4197061070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc10c5a0 msgr2=0x7fbccc1018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.810+0000 7fbcd235f640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc10c5a0 0x7fbccc1018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.810+0000 7fbcd235f640 1 -- 172.21.15.26:0/4197061070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc1ab0d0 con 0x7fbccc1031d0 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.810+0000 7fbcd235f640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1031d0 0x7fbccc100e20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005940 tx=0x7fbcc802d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.810+0000 7fbcc37fe640 1 -- 172.21.15.26:0/4197061070 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8043020 con 0x7fbccc1031d0 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.810+0000 7fbcd45c3640 1 -- 172.21.15.26:0/4197061070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc1ab2a0 con 0x7fbccc1031d0 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.810+0000 7fbcc37fe640 1 -- 172.21.15.26:0/4197061070 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcc802dc90 con 0x7fbccc1031d0 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.810+0000 7fbcc37fe640 1 -- 172.21.15.26:0/4197061070 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8003ab0 con 0x7fbccc1031d0 2024-04-21T10:31:06.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.810+0000 7fbcd45c3640 1 -- 172.21.15.26:0/4197061070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc1ab840 con 0x7fbccc1031d0 2024-04-21T10:31:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.811+0000 7fbcd45c3640 1 -- 172.21.15.26:0/4197061070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbccc06ae60 con 0x7fbccc1031d0 2024-04-21T10:31:06.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.812+0000 7fbcc37fe640 1 -- 172.21.15.26:0/4197061070 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fbcc8046020 con 0x7fbccc1031d0 2024-04-21T10:31:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.813+0000 7fbcc37fe640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fbcb80a3ed0 0x7fbcb80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.813+0000 7fbcd1b5e640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fbcb80a3ed0 0x7fbcb80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.814+0000 7fbcc37fe640 1 -- 172.21.15.26:0/4197061070 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fbcc80f3200 con 0x7fbccc1031d0 2024-04-21T10:31:06.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.814+0000 7fbcd1b5e640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fbcb80a3ed0 0x7fbcb80a6390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc005fd0 tx=0x7fbcbc005850 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:06.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:06.817+0000 7fbcc37fe640 1 -- 172.21.15.26:0/4197061070 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcc804d020 con 0x7fbccc1031d0 2024-04-21T10:31:06.823 INFO:tasks.workunit.client.0.smithi026.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-21T10:31:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.047+0000 7fbcd45c3640 1 -- 172.21.15.26:0/4197061070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbccc107690 con 0x7fbccc1031d0 2024-04-21T10:31:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.048+0000 7fbcc37fe640 1 -- 172.21.15.26:0/4197061070 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7fbcc80bf7c0 con 0x7fbccc1031d0 2024-04-21T10:31:07.047 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:07.047 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:30:56.186598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24478,24488]},"id":1}]} 2024-04-21T10:31:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 21 2024-04-21T10:31:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.050+0000 7fbcd45c3640 1 -- 172.21.15.26:0/4197061070 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fbcb80a3ed0 msgr2=0x7fbcb80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.050+0000 7fbcd45c3640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fbcb80a3ed0 0x7fbcb80a6390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc005fd0 tx=0x7fbcbc005850 comp rx=0 tx=0).stop 2024-04-21T10:31:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.050+0000 7fbcd45c3640 1 -- 172.21.15.26:0/4197061070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1031d0 msgr2=0x7fbccc100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.050+0000 7fbcd45c3640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1031d0 0x7fbccc100e20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005940 tx=0x7fbcc802d890 comp rx=0 tx=0).stop 2024-04-21T10:31:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.050+0000 7fbcd45c3640 1 -- 172.21.15.26:0/4197061070 shutdown_connections 2024-04-21T10:31:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.050+0000 7fbcd45c3640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fbcb80a3ed0 0x7fbcb80a6390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.050+0000 7fbcd45c3640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc10c5a0 0x7fbccc1018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.050+0000 7fbcd45c3640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbccc103af0 0x7fbccc101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.050+0000 7fbcd45c3640 1 --2- 172.21.15.26:0/4197061070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbccc1031d0 0x7fbccc100e20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.050+0000 7fbcd45c3640 1 -- 172.21.15.26:0/4197061070 >> 172.21.15.26:0/4197061070 conn(0x7fbccc0771d0 msgr2=0x7fbccc0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.050+0000 7fbcd45c3640 1 -- 172.21.15.26:0/4197061070 shutdown_connections 2024-04-21T10:31:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.050+0000 7fbcd45c3640 1 -- 172.21.15.26:0/4197061070 wait complete. 2024-04-21T10:31:07.060 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T10:29:01.278914+0000', 'modified': '2024-04-21T10:30:56.186598+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24496, 'mds_1': 24488, 'mds_2': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6835/1594119566', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 1594119566}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 1594119566}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6837/2325983283', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 2325983283}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 2325983283}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6835/1573483809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 1573483809}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 1573483809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.190:6837/2091448511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 2091448511}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 2091448511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 1, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6839/2428626465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 2428626465}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 2428626465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.190:6839/1574521545', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 1574521545}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 1574521545}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24496, 'qdb_cluster': [24496, 24478, 24488]} 2024-04-21T10:31:07.061 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-21T10:31:07.061 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-21T10:31:07.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.204+0000 7fdb0297e640 1 -- 172.21.15.26:0/1770931963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdafc100450 msgr2=0x7fdafc1008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.204+0000 7fdb0297e640 1 --2- 172.21.15.26:0/1770931963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdafc100450 0x7fdafc1008d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009930 tx=0x7fdaf002bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.204+0000 7fdb0297e640 1 -- 172.21.15.26:0/1770931963 shutdown_connections 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.204+0000 7fdb0297e640 1 --2- 172.21.15.26:0/1770931963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdafc100e10 0x7fdafc10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.204+0000 7fdb0297e640 1 --2- 172.21.15.26:0/1770931963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdafc100450 0x7fdafc1008d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.204+0000 7fdb0297e640 1 --2- 172.21.15.26:0/1770931963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdafc106f30 0x7fdafc107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.204+0000 7fdb0297e640 1 -- 172.21.15.26:0/1770931963 >> 172.21.15.26:0/1770931963 conn(0x7fdafc0773e0 msgr2=0x7fdafc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.204+0000 7fdb0297e640 1 -- 172.21.15.26:0/1770931963 shutdown_connections 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.204+0000 7fdb0297e640 1 -- 172.21.15.26:0/1770931963 wait complete. 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdb0297e640 1 Processor -- start 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdb0297e640 1 -- start start 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdb0297e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdafc100450 0x7fdafc101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdb0297e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdafc100e10 0x7fdafc101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdb0297e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdafc106f30 0x7fdafc1020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdb0297e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1122f0 con 0x7fdafc100450 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdb0297e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc112170 con 0x7fdafc100e10 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdb0297e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc112470 con 0x7fdafc106f30 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdafc100450 0x7fdafc101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdafc100450 0x7fdafc101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33224/0 (socket says 172.21.15.26:33224) 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdafbfff640 1 -- 172.21.15.26:0/2622682476 learned_addr learned my addr 172.21.15.26:0/2622682476 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdb00f1b640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdafc106f30 0x7fdafc1020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdafc100e10 0x7fdafc101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdafbfff640 1 -- 172.21.15.26:0/2622682476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdafc106f30 msgr2=0x7fdafc1020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdafbfff640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdafc106f30 0x7fdafc1020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdafbfff640 1 -- 172.21.15.26:0/2622682476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdafc100e10 msgr2=0x7fdafc101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdafbfff640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdafc100e10 0x7fdafc101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdafbfff640 1 -- 172.21.15.26:0/2622682476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc1a6c70 con 0x7fdafc100450 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.205+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdafc100e10 0x7fdafc101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.206+0000 7fdafbfff640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdafc100450 0x7fdafc101630 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00c920 tx=0x7fdaec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.206+0000 7fdaf97fa640 1 -- 172.21.15.26:0/2622682476 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec007c70 con 0x7fdafc100450 2024-04-21T10:31:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.206+0000 7fdaf97fa640 1 -- 172.21.15.26:0/2622682476 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaec007e10 con 0x7fdafc100450 2024-04-21T10:31:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.206+0000 7fdb0297e640 1 -- 172.21.15.26:0/2622682476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdafc1a6f60 con 0x7fdafc100450 2024-04-21T10:31:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.206+0000 7fdaf97fa640 1 -- 172.21.15.26:0/2622682476 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec018820 con 0x7fdafc100450 2024-04-21T10:31:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.206+0000 7fdb0297e640 1 -- 172.21.15.26:0/2622682476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdafc1a73d0 con 0x7fdafc100450 2024-04-21T10:31:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.208+0000 7fdaf97fa640 1 -- 172.21.15.26:0/2622682476 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fdaec00f450 con 0x7fdafc100450 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.208+0000 7fdb05569740 1 Processor -- start 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.208+0000 7fdb05569740 1 -- start start 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdb05569740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b447b0 0x559dc7b44b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdb05569740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b45160 0x559dc7b4d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdb05569740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b4dce0 0x559dc7b50130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdb05569740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559dc7aa8b10 con 0x559dc7b4dce0 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdb05569740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559dc7a46240 con 0x559dc7b447b0 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdb05569740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559dc7aa4ad0 con 0x559dc7b45160 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdafb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b4dce0 0x559dc7b50130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdaf97fa640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fdae40a3f00 0x7fdae40a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdafb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b4dce0 0x559dc7b50130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33230/0 (socket says 172.21.15.26:33230) 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdafb7fe640 1 -- 172.21.15.26:0/4267804569 learned_addr learned my addr 172.21.15.26:0/4267804569 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdafbfff640 1 --2- 172.21.15.26:0/4267804569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b45160 0x559dc7b4d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdb00f1b640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fdae40a3f00 0x7fdae40a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdb00f1b640 1 --2- 172.21.15.26:0/4267804569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b447b0 0x559dc7b44b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdaf97fa640 1 -- 172.21.15.26:0/2622682476 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fdaec0d0ca0 con 0x7fdafc100450 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdafb7fe640 1 -- 172.21.15.26:0/4267804569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b45160 msgr2=0x559dc7b4d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/4267804569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b45160 0x559dc7b4d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdafb7fe640 1 -- 172.21.15.26:0/4267804569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b447b0 msgr2=0x559dc7b44b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/4267804569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b447b0 0x559dc7b44b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdafb7fe640 1 -- 172.21.15.26:0/4267804569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dc77bb2d0 con 0x559dc7b4dce0 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.209+0000 7fdb00f1b640 1 --2- 172.21.15.26:0/4267804569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b447b0 0x559dc7b44b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/4267804569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b4dce0 0x559dc7b50130 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0036040 tx=0x7fdaf0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7240d334330a8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdb00f1b640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fdae40a3f00 0x7fdae40a63c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009fd0 tx=0x7fdaf4009540 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdae27fc640 1 -- 172.21.15.26:0/4267804569 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0044020 con 0x559dc7b4dce0 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdae27fc640 1 -- 172.21.15.26:0/4267804569 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf002cc90 con 0x559dc7b4dce0 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdae27fc640 1 -- 172.21.15.26:0/4267804569 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00048c0 con 0x559dc7b4dce0 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdb05569740 1 -- 172.21.15.26:0/4267804569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b4dce0 msgr2=0x559dc7b50130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdb05569740 1 --2- 172.21.15.26:0/4267804569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b4dce0 0x559dc7b50130 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0036040 tx=0x7fdaf0004040 comp rx=0 tx=0).stop 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdb05569740 1 -- 172.21.15.26:0/4267804569 shutdown_connections 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdb05569740 1 --2- 172.21.15.26:0/4267804569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b4dce0 0x559dc7b50130 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdb05569740 1 --2- 172.21.15.26:0/4267804569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b45160 0x559dc7b4d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdb05569740 1 --2- 172.21.15.26:0/4267804569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b447b0 0x559dc7b44b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdb05569740 1 -- 172.21.15.26:0/4267804569 >> 172.21.15.26:0/4267804569 conn(0x559dc7b3ea60 msgr2=0x559dc7b3ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdb05569740 1 -- 172.21.15.26:0/4267804569 shutdown_connections 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.210+0000 7fdb05569740 1 -- 172.21.15.26:0/4267804569 wait complete. 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdb05569740 1 -- start start 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdb05569740 1 Processor -- start 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdb05569740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b447b0 0x559dc7b4ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdb05569740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b45160 0x559dc7b4b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdb05569740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 0x559dc7b4b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdb05569740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559dc7b50bf0 con 0x559dc7b447b0 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdb05569740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559dc7a31190 con 0x559dc7b4dce0 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdb05569740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559dc7b50d70 con 0x559dc7b45160 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdb05569740 10 client.0 _ll_register_callbacks cb 0x559dc7a2c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b447b0 0x559dc7b4ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b447b0 0x559dc7b4ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33238/0 (socket says 172.21.15.26:33238) 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdafbfff640 1 -- 172.21.15.26:0/863843827 learned_addr learned my addr 172.21.15.26:0/863843827 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b45160 0x559dc7b4b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.211+0000 7fdb00f1b640 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 0x559dc7b4b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdafbfff640 1 -- 172.21.15.26:0/863843827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b45160 msgr2=0x559dc7b4b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdafbfff640 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b45160 0x559dc7b4b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdafbfff640 1 -- 172.21.15.26:0/863843827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 msgr2=0x559dc7b4b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdafbfff640 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 0x559dc7b4b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdafbfff640 1 -- 172.21.15.26:0/863843827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dc78856b0 con 0x559dc7b447b0 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdb00f1b640 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 0x559dc7b4b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdafbfff640 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b447b0 0x559dc7b4ac30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdaec098e70 tx=0x7fdaec06f500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b32cbccbaf9e258 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdadb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdadb7fe640 1 -- 172.21.15.26:0/863843827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00de20 con 0x559dc7b447b0 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdadb7fe640 1 -- 172.21.15.26:0/863843827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fdaec099080 con 0x559dc7b447b0 2024-04-21T10:31:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdb05569740 1 -- 172.21.15.26:0/863843827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf002cc90 con 0x559dc7b447b0 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdadb7fe640 1 -- 172.21.15.26:0/863843827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdaec09ca10 con 0x559dc7b447b0 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdb05569740 10 client.15522 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdb05569740 1 -- 172.21.15.26:0/863843827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559dc7b5ce90 con 0x559dc7b447b0 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdadb7fe640 1 -- 172.21.15.26:0/863843827 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf002cc90 con 0x559dc7b447b0 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdadb7fe640 1 -- 172.21.15.26:0/863843827 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdaec0a51f0 con 0x559dc7b447b0 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdb05569740 10 client.15522 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdb05569740 10 client.15522 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.212+0000 7fdb05569740 20 client.15522 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.213+0000 7fdb05569740 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x559dc7b5f4e0 0x559dc7b60950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.213+0000 7fdb05569740 4 client.15522 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.213+0000 7fdb05569740 1 -- 172.21.15.26:0/863843827 --> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dc7ab6530 con 0x559dc7b5f4e0 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.213+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x559dc7b5f4e0 0x559dc7b60950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.213+0000 7fdadb7fe640 10 client.15522 ms_handle_connect on v2:172.21.15.190:6836/2091448511 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.213+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x559dc7b5f4e0 0x559dc7b60950 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85b06bf86c41b402 server_cookie=a91340843996ecc6 in_seq=0 out_seq=0 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdadb7fe640 1 -- 172.21.15.26:0/863843827 <== mds.2 v2:172.21.15.190:6836/2091448511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559dc7ab6530 con 0x559dc7b5f4e0 2024-04-21T10:31:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdadb7fe640 10 client.15522 handle_command_reply: tid=0 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdb05569740 1 client.15522 shutdown 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdb05569740 1 -- 172.21.15.26:0/863843827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b447b0 msgr2=0x559dc7b4ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdb05569740 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b447b0 0x559dc7b4ac30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdaec098e70 tx=0x7fdaec06f500 comp rx=0 tx=0).stop 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdb05569740 1 -- 172.21.15.26:0/863843827 shutdown_connections 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdb05569740 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 0x559dc7b4b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdb05569740 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b45160 0x559dc7b4b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdb05569740 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x559dc7b5f4e0 0x559dc7b60950 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdb05569740 1 --2- 172.21.15.26:0/863843827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b447b0 0x559dc7b4ac30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdadb7fe640 0 client.15522 ms_handle_reset on v2:172.21.15.190:6836/2091448511 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdb05569740 1 -- 172.21.15.26:0/863843827 >> 172.21.15.26:0/863843827 conn(0x559dc7b3ea60 msgr2=0x559dc7b40bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdb05569740 1 -- 172.21.15.26:0/863843827 shutdown_connections 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdb05569740 1 -- 172.21.15.26:0/863843827 wait complete. 2024-04-21T10:31:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.214+0000 7fdb05569740 20 client.15522 trim_cache size 0 max 16384 2024-04-21T10:31:07.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdb05569740 1 Processor -- start 2024-04-21T10:31:07.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdb05569740 1 -- start start 2024-04-21T10:31:07.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdb05569740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b447b0 0x559dc7b45ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdb05569740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 0x559dc7b46430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdb05569740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 0x559dc7b49500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdb05569740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559dc7b5ec90 con 0x559dc7b45160 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdb05569740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559dc7b5f360 con 0x559dc7b4dce0 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdb05569740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559dc7b5fa30 con 0x559dc7b447b0 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdafb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 0x559dc7b46430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdafb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 0x559dc7b46430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33252/0 (socket says 172.21.15.26:33252) 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdafb7fe640 1 -- 172.21.15.26:0/3582097493 learned_addr learned my addr 172.21.15.26:0/3582097493 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdafb7fe640 1 -- 172.21.15.26:0/3582097493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b447b0 msgr2=0x559dc7b45ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/3582097493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b447b0 0x559dc7b45ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdafb7fe640 1 -- 172.21.15.26:0/3582097493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 msgr2=0x559dc7b49500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/3582097493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 0x559dc7b49500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.233+0000 7fdafb7fe640 1 -- 172.21.15.26:0/3582097493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dc7b500d0 con 0x559dc7b45160 2024-04-21T10:31:07.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.234+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/3582097493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 0x559dc7b46430 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdaf002c170 tx=0x7fdaf0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f6b540da131a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.234+0000 7fdae1ffb640 1 -- 172.21.15.26:0/3582097493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0044040 con 0x559dc7b45160 2024-04-21T10:31:07.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.234+0000 7fdae1ffb640 1 -- 172.21.15.26:0/3582097493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf002cc90 con 0x559dc7b45160 2024-04-21T10:31:07.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.234+0000 7fdae1ffb640 1 -- 172.21.15.26:0/3582097493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00035f0 con 0x559dc7b45160 2024-04-21T10:31:07.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.234+0000 7fdb05569740 1 -- 172.21.15.26:0/3582097493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 msgr2=0x559dc7b46430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.234+0000 7fdb05569740 1 --2- 172.21.15.26:0/3582097493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 0x559dc7b46430 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdaf002c170 tx=0x7fdaf0005fd0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.234+0000 7fdb05569740 1 -- 172.21.15.26:0/3582097493 shutdown_connections 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.234+0000 7fdb05569740 1 --2- 172.21.15.26:0/3582097493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 0x559dc7b49500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.234+0000 7fdb05569740 1 --2- 172.21.15.26:0/3582097493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 0x559dc7b46430 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.234+0000 7fdb05569740 1 --2- 172.21.15.26:0/3582097493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b447b0 0x559dc7b45ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.234+0000 7fdb05569740 1 -- 172.21.15.26:0/3582097493 >> 172.21.15.26:0/3582097493 conn(0x559dc7b3ea60 msgr2=0x559dc7b39800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdb05569740 1 -- 172.21.15.26:0/3582097493 shutdown_connections 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdb05569740 1 -- 172.21.15.26:0/3582097493 wait complete. 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdb05569740 1 -- start start 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdb05569740 1 Processor -- start 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdb05569740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b447b0 0x559dc7b62520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdb05569740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 0x559dc7b62a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdb05569740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 0x559dc7b62fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdb05569740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559dc7b607d0 con 0x559dc7b45160 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdb05569740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559dc7b60100 con 0x559dc7b4dce0 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdb05569740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559dc7b60ea0 con 0x559dc7b447b0 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdb05569740 10 client.0 _ll_register_callbacks cb 0x7fdaec085f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdafb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 0x559dc7b62a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdafb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 0x559dc7b62a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33254/0 (socket says 172.21.15.26:33254) 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdafb7fe640 1 -- 172.21.15.26:0/4291114611 learned_addr learned my addr 172.21.15.26:0/4291114611 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.235+0000 7fdafbfff640 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b447b0 0x559dc7b62520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.236+0000 7fdb00f1b640 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 0x559dc7b62fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.236+0000 7fdafb7fe640 1 -- 172.21.15.26:0/4291114611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b447b0 msgr2=0x559dc7b62520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.236+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b447b0 0x559dc7b62520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.236+0000 7fdafb7fe640 1 -- 172.21.15.26:0/4291114611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 msgr2=0x559dc7b62fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.236+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 0x559dc7b62fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.236+0000 7fdafb7fe640 1 -- 172.21.15.26:0/4291114611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dc7b4aa00 con 0x559dc7b45160 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.236+0000 7fdafb7fe640 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 0x559dc7b62a60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x559dc7b39f30 tx=0x7fdaf00538a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350f1e5992e74c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.236+0000 7fdadb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.236+0000 7fdadb7fe640 1 -- 172.21.15.26:0/4291114611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf004d070 con 0x559dc7b45160 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.236+0000 7fdadb7fe640 1 -- 172.21.15.26:0/4291114611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fdaf0053400 con 0x559dc7b45160 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.236+0000 7fdb05569740 1 -- 172.21.15.26:0/4291114611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf002cc90 con 0x559dc7b45160 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.237+0000 7fdadb7fe640 1 -- 172.21.15.26:0/4291114611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdaf0044040 con 0x559dc7b45160 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.237+0000 7fdb05569740 10 client.15528 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.237+0000 7fdb05569740 1 -- 172.21.15.26:0/4291114611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559dc7bc5490 con 0x559dc7b45160 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.237+0000 7fdadb7fe640 1 -- 172.21.15.26:0/4291114611 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf002cc90 con 0x559dc7b45160 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.237+0000 7fdadb7fe640 1 -- 172.21.15.26:0/4291114611 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdaf0040c50 con 0x559dc7b45160 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.237+0000 7fdb05569740 10 client.15528 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.237+0000 7fdb05569740 10 client.15528 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.237+0000 7fdb05569740 20 client.15528 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.237+0000 7fdb05569740 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x559dc7bc9950 0x559dc7bc8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.237+0000 7fdb05569740 4 client.15528 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.237+0000 7fdb05569740 1 -- 172.21.15.26:0/4291114611 --> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] -- command(tid 0: {"prefix": "status"}) -- 0x559dc7ab03a0 con 0x559dc7bc9950 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.237+0000 7fdafbfff640 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x559dc7bc9950 0x559dc7bc8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdadb7fe640 10 client.15528 ms_handle_connect on v2:172.21.15.190:6836/2091448511 2024-04-21T10:31:07.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdafbfff640 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x559dc7bc9950 0x559dc7bc8660 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e8b35d1b0c87619 server_cookie=61a62182bd2d23a1 in_seq=0 out_seq=0 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdadb7fe640 1 -- 172.21.15.26:0/4291114611 <== mds.2 v2:172.21.15.190:6836/2091448511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x559dc7ab03a0 con 0x559dc7bc9950 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdadb7fe640 10 client.15528 handle_command_reply: tid=0 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdb05569740 1 client.15528 shutdown 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdb05569740 1 -- 172.21.15.26:0/4291114611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 msgr2=0x559dc7b62a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdb05569740 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 0x559dc7b62a60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x559dc7b39f30 tx=0x7fdaf00538a0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdb05569740 1 -- 172.21.15.26:0/4291114611 shutdown_connections 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdb05569740 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559dc7b4dce0 0x559dc7b62fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdb05569740 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.190:6836/2091448511,v1:172.21.15.190:6837/2091448511] conn(0x559dc7bc9950 0x559dc7bc8660 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdb05569740 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559dc7b45160 0x559dc7b62a60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdb05569740 1 --2- 172.21.15.26:0/4291114611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dc7b447b0 0x559dc7b62520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdadb7fe640 0 client.15528 ms_handle_reset on v2:172.21.15.190:6836/2091448511 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdb05569740 1 -- 172.21.15.26:0/4291114611 >> 172.21.15.26:0/4291114611 conn(0x559dc7b3ea60 msgr2=0x559dc7b45ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdb05569740 1 -- 172.21.15.26:0/4291114611 shutdown_connections 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdb05569740 1 -- 172.21.15.26:0/4291114611 wait complete. 2024-04-21T10:31:07.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.238+0000 7fdb05569740 20 client.15528 trim_cache size 0 max 16384 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24478, 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 122.435631497, 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 21, 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 90, 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 144.753305797 2024-04-21T10:31:07.243 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:31:07.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.239+0000 7fdb0297e640 1 -- 172.21.15.26:0/2622682476 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fdae40a3f00 msgr2=0x7fdae40a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.239+0000 7fdb0297e640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fdae40a3f00 0x7fdae40a63c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009fd0 tx=0x7fdaf4009540 comp rx=0 tx=0).stop 2024-04-21T10:31:07.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.239+0000 7fdb0297e640 1 -- 172.21.15.26:0/2622682476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdafc100450 msgr2=0x7fdafc101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.239+0000 7fdb0297e640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdafc100450 0x7fdafc101630 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00c920 tx=0x7fdaec00cde0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.239+0000 7fdb0297e640 1 -- 172.21.15.26:0/2622682476 shutdown_connections 2024-04-21T10:31:07.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.239+0000 7fdb0297e640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fdae40a3f00 0x7fdae40a63c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.239+0000 7fdb0297e640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdafc106f30 0x7fdafc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.239+0000 7fdb0297e640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdafc100e10 0x7fdafc101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.239+0000 7fdb0297e640 1 --2- 172.21.15.26:0/2622682476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdafc100450 0x7fdafc101630 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.239+0000 7fdb0297e640 1 -- 172.21.15.26:0/2622682476 >> 172.21.15.26:0/2622682476 conn(0x7fdafc0773e0 msgr2=0x7fdafc104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.239+0000 7fdb0297e640 1 -- 172.21.15.26:0/2622682476 shutdown_connections 2024-04-21T10:31:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.239+0000 7fdb0297e640 1 -- 172.21.15.26:0/2622682476 wait complete. 2024-04-21T10:31:07.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T10:31:07.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 -- 172.21.15.26:0/450262538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a48107270 msgr2=0x7f0a4810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 --2- 172.21.15.26:0/450262538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a48107270 0x7f0a4810ee40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009ca0 tx=0x7f0a4002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 -- 172.21.15.26:0/450262538 shutdown_connections 2024-04-21T10:31:07.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 --2- 172.21.15.26:0/450262538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a48107270 0x7f0a4810ee40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 --2- 172.21.15.26:0/450262538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a4806b1f0 0x7f0a48106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 --2- 172.21.15.26:0/450262538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a4806a840 0x7f0a4806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 -- 172.21.15.26:0/450262538 >> 172.21.15.26:0/450262538 conn(0x7f0a48077eb0 msgr2=0x7f0a480782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 -- 172.21.15.26:0/450262538 shutdown_connections 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 -- 172.21.15.26:0/450262538 wait complete. 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 Processor -- start 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 -- start start 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a4806a840 0x7f0a48100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a48107270 0x7f0a481013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a48101900 0x7f0a481a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481121d0 con 0x7f0a48107270 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48112050 con 0x7f0a48101900 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.353+0000 7f0a4ec69640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48112350 con 0x7f0a4806a840 2024-04-21T10:31:07.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a48107270 0x7f0a481013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a48107270 0x7f0a481013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33272/0 (socket says 172.21.15.26:33272) 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a47fff640 1 -- 172.21.15.26:0/2882434315 learned_addr learned my addr 172.21.15.26:0/2882434315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a4ca05640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a4806a840 0x7f0a48100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a4d206640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a48101900 0x7f0a481a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a47fff640 1 -- 172.21.15.26:0/2882434315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a4806a840 msgr2=0x7f0a48100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a47fff640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a4806a840 0x7f0a48100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a47fff640 1 -- 172.21.15.26:0/2882434315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a48101900 msgr2=0x7f0a481a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a47fff640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a48101900 0x7f0a481a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a47fff640 1 -- 172.21.15.26:0/2882434315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a481a6f80 con 0x7f0a48107270 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a47fff640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a48107270 0x7f0a481013c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400e9f0 tx=0x7f0a3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a45ffb640 1 -- 172.21.15.26:0/2882434315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3400ccc0 con 0x7f0a48107270 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a4ec69640 1 -- 172.21.15.26:0/2882434315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a481a7270 con 0x7f0a48107270 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a45ffb640 1 -- 172.21.15.26:0/2882434315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a340054d0 con 0x7f0a48107270 2024-04-21T10:31:07.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a45ffb640 1 -- 172.21.15.26:0/2882434315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34010430 con 0x7f0a48107270 2024-04-21T10:31:07.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.354+0000 7f0a4ec69640 1 -- 172.21.15.26:0/2882434315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a481a7770 con 0x7f0a48107270 2024-04-21T10:31:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.356+0000 7f0a45ffb640 1 -- 172.21.15.26:0/2882434315 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f0a340046c0 con 0x7f0a48107270 2024-04-21T10:31:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.357+0000 7f0a51857740 1 Processor -- start 2024-04-21T10:31:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.357+0000 7f0a51857740 1 -- start start 2024-04-21T10:31:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.357+0000 7f0a45ffb640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0a380a3f00 0x7f0a380a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.357+0000 7f0a51857740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039d8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.357+0000 7f0a51857740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d9160 0x5636039e1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.357+0000 7f0a51857740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039e4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.357+0000 7f0a51857740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56360393cb10 con 0x5636039e1ce0 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.357+0000 7f0a51857740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5636038da240 con 0x5636039d9160 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.357+0000 7f0a51857740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x563603938ad0 con 0x5636039d87b0 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a4d206640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0a380a3f00 0x7f0a380a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a45ffb640 1 -- 172.21.15.26:0/2882434315 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f0a340d0da0 con 0x7f0a48107270 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a4ca05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039d8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a4ca05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039d8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.26:35892/0 (socket says 172.21.15.26:35892) 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a4ca05640 1 -- 172.21.15.26:0/955643680 learned_addr learned my addr 172.21.15.26:0/955643680 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a4d206640 1 --2- 172.21.15.26:0/955643680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039e4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a47fff640 1 --2- 172.21.15.26:0/955643680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d9160 0x5636039e1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a4d206640 1 -- 172.21.15.26:0/955643680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 msgr2=0x5636039d8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a4d206640 1 --2- 172.21.15.26:0/955643680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039d8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a4d206640 1 -- 172.21.15.26:0/955643680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d9160 msgr2=0x5636039e1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a4d206640 1 --2- 172.21.15.26:0/955643680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d9160 0x5636039e1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a4d206640 1 -- 172.21.15.26:0/955643680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56360364f2d0 con 0x5636039e1ce0 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a4d206640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0a380a3f00 0x7f0a380a63c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0a40004af0 tx=0x7f0a400099f0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a4d206640 1 --2- 172.21.15.26:0/955643680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039e4130 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0a4002c700 tx=0x7f0a40047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a92b0fbfef9e2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.358+0000 7f0a32ffd640 1 -- 172.21.15.26:0/955643680 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40039040 con 0x5636039e1ce0 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.359+0000 7f0a32ffd640 1 -- 172.21.15.26:0/955643680 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40003e20 con 0x5636039e1ce0 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.359+0000 7f0a32ffd640 1 -- 172.21.15.26:0/955643680 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40004570 con 0x5636039e1ce0 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.359+0000 7f0a51857740 1 -- 172.21.15.26:0/955643680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 msgr2=0x5636039e4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.359+0000 7f0a51857740 1 --2- 172.21.15.26:0/955643680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039e4130 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0a4002c700 tx=0x7f0a40047000 comp rx=0 tx=0).stop 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.359+0000 7f0a51857740 1 -- 172.21.15.26:0/955643680 shutdown_connections 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.359+0000 7f0a51857740 1 --2- 172.21.15.26:0/955643680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039e4130 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.359+0000 7f0a51857740 1 --2- 172.21.15.26:0/955643680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d9160 0x5636039e1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.359+0000 7f0a51857740 1 --2- 172.21.15.26:0/955643680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039d8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.359+0000 7f0a51857740 1 -- 172.21.15.26:0/955643680 >> 172.21.15.26:0/955643680 conn(0x5636039d2a60 msgr2=0x5636039d2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.359+0000 7f0a51857740 1 -- 172.21.15.26:0/955643680 shutdown_connections 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.359+0000 7f0a51857740 1 -- 172.21.15.26:0/955643680 wait complete. 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a51857740 1 -- start start 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a51857740 1 Processor -- start 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a51857740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039decb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a51857740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d9160 0x5636039df1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a51857740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039df730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a51857740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5636039e4bf0 con 0x5636039e1ce0 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a51857740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5636038c5190 con 0x5636039d9160 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a51857740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5636039e4d70 con 0x5636039d87b0 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a51857740 10 client.0 _ll_register_callbacks cb 0x5636038c08b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a4d206640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039df730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a4d206640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039df730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33280/0 (socket says 172.21.15.26:33280) 2024-04-21T10:31:07.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a4d206640 1 -- 172.21.15.26:0/373649294 learned_addr learned my addr 172.21.15.26:0/373649294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a47fff640 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d9160 0x5636039df1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a4ca05640 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039decb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a4d206640 1 -- 172.21.15.26:0/373649294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 msgr2=0x5636039decb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.360+0000 7f0a4d206640 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039decb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a4d206640 1 -- 172.21.15.26:0/373649294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d9160 msgr2=0x5636039df1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a4d206640 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d9160 0x5636039df1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a4d206640 1 -- 172.21.15.26:0/373649294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636037196b0 con 0x5636039e1ce0 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a47fff640 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d9160 0x5636039df1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a4d206640 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039df730 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0a40006010 tx=0x7f0a40004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4420a6eb66dbcd68 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a27fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a27fff640 1 -- 172.21.15.26:0/373649294 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4005b060 con 0x5636039e1ce0 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a27fff640 1 -- 172.21.15.26:0/373649294 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f0a40004000 con 0x5636039e1ce0 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a51857740 1 -- 172.21.15.26:0/373649294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a40003e20 con 0x5636039e1ce0 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a27fff640 1 -- 172.21.15.26:0/373649294 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0a40039040 con 0x5636039e1ce0 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a51857740 10 client.15540 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a51857740 1 -- 172.21.15.26:0/373649294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5636039e4bf0 con 0x5636039e1ce0 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a27fff640 1 -- 172.21.15.26:0/373649294 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40003e20 con 0x5636039e1ce0 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a27fff640 1 -- 172.21.15.26:0/373649294 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0a4003f070 con 0x5636039e1ce0 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a51857740 10 client.15540 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.361+0000 7f0a51857740 10 client.15540 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.362+0000 7f0a51857740 20 client.15540 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.362+0000 7f0a51857740 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x5636039f2fb0 0x5636039f4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.362+0000 7f0a51857740 4 client.15540 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.362+0000 7f0a51857740 1 -- 172.21.15.26:0/373649294 --> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56360394a530 con 0x5636039f2fb0 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.362+0000 7f0a4ca05640 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x5636039f2fb0 0x5636039f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.362+0000 7f0a27fff640 10 client.15540 ms_handle_connect on v2:172.21.15.169:6838/2428626465 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.362+0000 7f0a4ca05640 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x5636039f2fb0 0x5636039f4420 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1205d34a30ce4587 server_cookie=8a3f8a2bfa14ce12 in_seq=0 out_seq=0 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a27fff640 1 -- 172.21.15.26:0/373649294 <== mds.1 v2:172.21.15.169:6838/2428626465 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56360394a530 con 0x5636039f2fb0 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a27fff640 10 client.15540 handle_command_reply: tid=0 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a51857740 1 client.15540 shutdown 2024-04-21T10:31:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a51857740 1 -- 172.21.15.26:0/373649294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 msgr2=0x5636039df730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a51857740 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039df730 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0a40006010 tx=0x7f0a40004df0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a51857740 1 -- 172.21.15.26:0/373649294 shutdown_connections 2024-04-21T10:31:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a51857740 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x5636039f2fb0 0x5636039f4420 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a51857740 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039df730 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a51857740 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d9160 0x5636039df1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a27fff640 0 client.15540 ms_handle_reset on v2:172.21.15.169:6838/2428626465 2024-04-21T10:31:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a51857740 1 --2- 172.21.15.26:0/373649294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039decb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a51857740 1 -- 172.21.15.26:0/373649294 >> 172.21.15.26:0/373649294 conn(0x5636039d2a60 msgr2=0x5636039d4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a51857740 1 -- 172.21.15.26:0/373649294 shutdown_connections 2024-04-21T10:31:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a51857740 1 -- 172.21.15.26:0/373649294 wait complete. 2024-04-21T10:31:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.363+0000 7f0a51857740 20 client.15540 trim_cache size 0 max 16384 2024-04-21T10:31:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a51857740 1 Processor -- start 2024-04-21T10:31:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a51857740 1 -- start start 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a51857740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d87b0 0x5636039df550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a51857740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d9160 0x5636039dbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a51857740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039dc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a51857740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5636039f2760 con 0x5636039e1ce0 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a51857740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5636039f2e30 con 0x5636039d87b0 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a51857740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5636039f3500 con 0x5636039d9160 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a4d206640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039dc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a4d206640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039dc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33290/0 (socket says 172.21.15.26:33290) 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a4d206640 1 -- 172.21.15.26:0/991173720 learned_addr learned my addr 172.21.15.26:0/991173720 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a47fff640 1 --2- 172.21.15.26:0/991173720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d9160 0x5636039dbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a4ca05640 1 --2- 172.21.15.26:0/991173720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d87b0 0x5636039df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a4d206640 1 -- 172.21.15.26:0/991173720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d9160 msgr2=0x5636039dbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a4d206640 1 --2- 172.21.15.26:0/991173720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d9160 0x5636039dbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a4d206640 1 -- 172.21.15.26:0/991173720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d87b0 msgr2=0x5636039df550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a4d206640 1 --2- 172.21.15.26:0/991173720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d87b0 0x5636039df550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.382+0000 7f0a4d206640 1 -- 172.21.15.26:0/991173720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636039dd200 con 0x5636039e1ce0 2024-04-21T10:31:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.383+0000 7f0a4d206640 1 --2- 172.21.15.26:0/991173720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039dc450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0a40033000 tx=0x7f0a400329f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc84cc02b2cbf48e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.383+0000 7f0a327fc640 1 -- 172.21.15.26:0/991173720 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4005b040 con 0x5636039e1ce0 2024-04-21T10:31:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.383+0000 7f0a327fc640 1 -- 172.21.15.26:0/991173720 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40009b20 con 0x5636039e1ce0 2024-04-21T10:31:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.383+0000 7f0a327fc640 1 -- 172.21.15.26:0/991173720 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400047e0 con 0x5636039e1ce0 2024-04-21T10:31:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.383+0000 7f0a51857740 1 -- 172.21.15.26:0/991173720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 msgr2=0x5636039dc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.383+0000 7f0a51857740 1 --2- 172.21.15.26:0/991173720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039dc450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0a40033000 tx=0x7f0a400329f0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.383+0000 7f0a51857740 1 -- 172.21.15.26:0/991173720 shutdown_connections 2024-04-21T10:31:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.383+0000 7f0a51857740 1 --2- 172.21.15.26:0/991173720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039e1ce0 0x5636039dc450 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.383+0000 7f0a51857740 1 --2- 172.21.15.26:0/991173720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d9160 0x5636039dbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.383+0000 7f0a51857740 1 --2- 172.21.15.26:0/991173720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039d87b0 0x5636039df550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.383+0000 7f0a51857740 1 -- 172.21.15.26:0/991173720 >> 172.21.15.26:0/991173720 conn(0x5636039d2a60 msgr2=0x5636039cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a51857740 1 -- 172.21.15.26:0/991173720 shutdown_connections 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a51857740 1 -- 172.21.15.26:0/991173720 wait complete. 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a51857740 1 -- start start 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a51857740 1 Processor -- start 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a51857740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039f6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a51857740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039d9160 0x5636039f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a51857740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039e1ce0 0x5636039fae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a51857740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5636039f42a0 con 0x5636039d9160 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a51857740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5636039f3bd0 con 0x5636039e1ce0 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a51857740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5636039f4970 con 0x5636039d87b0 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a51857740 10 client.0 _ll_register_callbacks cb 0x7f0a3c00a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039d9160 0x5636039f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a47fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039d9160 0x5636039f6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33306/0 (socket says 172.21.15.26:33306) 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a47fff640 1 -- 172.21.15.26:0/1685847006 learned_addr learned my addr 172.21.15.26:0/1685847006 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a4ca05640 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.384+0000 7f0a4d206640 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039e1ce0 0x5636039fae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a47fff640 1 -- 172.21.15.26:0/1685847006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 msgr2=0x5636039f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a47fff640 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a47fff640 1 -- 172.21.15.26:0/1685847006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039e1ce0 msgr2=0x5636039fae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a47fff640 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039e1ce0 0x5636039fae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a47fff640 1 -- 172.21.15.26:0/1685847006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636039fb370 con 0x5636039d9160 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a47fff640 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039d9160 0x5636039f6d70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c6a0 tx=0x7f0a3409c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b15b7bd314dd52f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a27fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a27fff640 1 -- 172.21.15.26:0/1685847006 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34005680 con 0x5636039d9160 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a27fff640 1 -- 172.21.15.26:0/1685847006 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f0a340809d0 con 0x5636039d9160 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a51857740 1 -- 172.21.15.26:0/1685847006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a40009b20 con 0x5636039d9160 2024-04-21T10:31:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a27fff640 1 -- 172.21.15.26:0/1685847006 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0a3409d210 con 0x5636039d9160 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a51857740 10 client.15546 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a51857740 1 -- 172.21.15.26:0/1685847006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563603a59800 con 0x5636039d9160 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.385+0000 7f0a27fff640 1 -- 172.21.15.26:0/1685847006 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40009b20 con 0x5636039d9160 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.386+0000 7f0a27fff640 1 -- 172.21.15.26:0/1685847006 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f0a34084af0 con 0x5636039d9160 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.386+0000 7f0a51857740 10 client.15546 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.386+0000 7f0a51857740 10 client.15546 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.386+0000 7f0a51857740 20 client.15546 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.386+0000 7f0a51857740 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x563603a5e3e0 0x563603a5d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.386+0000 7f0a51857740 4 client.15546 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.386+0000 7f0a51857740 1 -- 172.21.15.26:0/1685847006 --> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] -- command(tid 0: {"prefix": "status"}) -- 0x5636039443a0 con 0x563603a5e3e0 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.386+0000 7f0a4ca05640 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x563603a5e3e0 0x563603a5d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.386+0000 7f0a27fff640 10 client.15546 ms_handle_connect on v2:172.21.15.169:6838/2428626465 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.386+0000 7f0a4ca05640 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x563603a5e3e0 0x563603a5d0f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=682bc07d00274e3b server_cookie=611e956c394f2115 in_seq=0 out_seq=0 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a27fff640 1 -- 172.21.15.26:0/1685847006 <== mds.1 v2:172.21.15.169:6838/2428626465 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5636039443a0 con 0x563603a5e3e0 2024-04-21T10:31:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a27fff640 10 client.15546 handle_command_reply: tid=0 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a51857740 1 client.15546 shutdown 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a51857740 1 -- 172.21.15.26:0/1685847006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039d9160 msgr2=0x5636039f6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a51857740 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039d9160 0x5636039f6d70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c6a0 tx=0x7f0a3409c6e0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a51857740 1 -- 172.21.15.26:0/1685847006 shutdown_connections 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a51857740 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.169:6838/2428626465,v1:172.21.15.169:6839/2428626465] conn(0x563603a5e3e0 0x563603a5d0f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a51857740 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636039e1ce0 0x5636039fae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a51857740 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636039d9160 0x5636039f6d70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a51857740 1 --2- 172.21.15.26:0/1685847006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5636039d87b0 0x5636039f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a27fff640 0 client.15546 ms_handle_reset on v2:172.21.15.169:6838/2428626465 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a51857740 1 -- 172.21.15.26:0/1685847006 >> 172.21.15.26:0/1685847006 conn(0x5636039d2a60 msgr2=0x5636039d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a51857740 1 -- 172.21.15.26:0/1685847006 shutdown_connections 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a51857740 1 -- 172.21.15.26:0/1685847006 wait complete. 2024-04-21T10:31:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a51857740 20 client.15546 trim_cache size 0 max 16384 2024-04-21T10:31:07.393 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:31:07.393 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:31:07.393 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2024-04-21T10:31:07.393 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24488, 2024-04-21T10:31:07.393 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-21T10:31:07.393 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-21T10:31:07.393 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-21T10:31:07.393 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 123.591647389, 2024-04-21T10:31:07.393 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 21, 2024-04-21T10:31:07.393 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 90, 2024-04-21T10:31:07.393 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T10:31:07.393 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 131.06168103900001 2024-04-21T10:31:07.394 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:31:07.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a4ec69640 1 -- 172.21.15.26:0/2882434315 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0a380a3f00 msgr2=0x7f0a380a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a4ec69640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0a380a3f00 0x7f0a380a63c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0a40004af0 tx=0x7f0a400099f0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a4ec69640 1 -- 172.21.15.26:0/2882434315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a48107270 msgr2=0x7f0a481013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a4ec69640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a48107270 0x7f0a481013c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400e9f0 tx=0x7f0a3400eeb0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.387+0000 7f0a4ec69640 1 -- 172.21.15.26:0/2882434315 shutdown_connections 2024-04-21T10:31:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.388+0000 7f0a4ec69640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f0a380a3f00 0x7f0a380a63c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.388+0000 7f0a4ec69640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a48101900 0x7f0a481a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.388+0000 7f0a4ec69640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a48107270 0x7f0a481013c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.388+0000 7f0a4ec69640 1 --2- 172.21.15.26:0/2882434315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a4806a840 0x7f0a48100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.388+0000 7f0a4ec69640 1 -- 172.21.15.26:0/2882434315 >> 172.21.15.26:0/2882434315 conn(0x7f0a48077eb0 msgr2=0x7f0a48100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.388+0000 7f0a4ec69640 1 -- 172.21.15.26:0/2882434315 shutdown_connections 2024-04-21T10:31:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.388+0000 7f0a4ec69640 1 -- 172.21.15.26:0/2882434315 wait complete. 2024-04-21T10:31:07.397 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:07 smithi026 ceph-mon[27965]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 170 KiB/s wr, 103 op/s 2024-04-21T10:31:07.397 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:07 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/89724914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:07.397 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:07 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/837684328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:31:07.397 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:07 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/4197061070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:07.398 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T10:31:07.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.501+0000 7f2f61248640 1 -- 172.21.15.26:0/3286819724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5c101090 msgr2=0x7f2f5c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.501+0000 7f2f61248640 1 --2- 172.21.15.26:0/3286819724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5c101090 0x7f2f5c101470 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009900 tx=0x7f2f4c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.501+0000 7f2f61248640 1 -- 172.21.15.26:0/3286819724 shutdown_connections 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.501+0000 7f2f61248640 1 --2- 172.21.15.26:0/3286819724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f5c106160 0x7f2f5c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.501+0000 7f2f61248640 1 --2- 172.21.15.26:0/3286819724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f5c101a40 0x7f2f5c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.501+0000 7f2f61248640 1 --2- 172.21.15.26:0/3286819724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5c101090 0x7f2f5c101470 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.501+0000 7f2f61248640 1 -- 172.21.15.26:0/3286819724 >> 172.21.15.26:0/3286819724 conn(0x7f2f5c077f50 msgr2=0x7f2f5c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.501+0000 7f2f61248640 1 -- 172.21.15.26:0/3286819724 shutdown_connections 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.501+0000 7f2f61248640 1 -- 172.21.15.26:0/3286819724 wait complete. 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.501+0000 7f2f61248640 1 Processor -- start 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.501+0000 7f2f61248640 1 -- start start 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f61248640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f5c101090 0x7f2f5c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f61248640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5c106160 0x7f2f5c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f61248640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f5c199fd0 0x7f2f5c1a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f61248640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c1110a0 con 0x7f2f5c106160 2024-04-21T10:31:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f61248640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c110f20 con 0x7f2f5c101090 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f61248640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5c111220 con 0x7f2f5c199fd0 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f53fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5c106160 0x7f2f5c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f53fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5c106160 0x7f2f5c199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33328/0 (socket says 172.21.15.26:33328) 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f53fff640 1 -- 172.21.15.26:0/3657234909 learned_addr learned my addr 172.21.15.26:0/3657234909 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f5b59e640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f5c199fd0 0x7f2f5c1a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f53fff640 1 -- 172.21.15.26:0/3657234909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f5c199fd0 msgr2=0x7f2f5c1a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f53fff640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f5c199fd0 0x7f2f5c1a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f53fff640 1 -- 172.21.15.26:0/3657234909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f5c101090 msgr2=0x7f2f5c199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f5ad9d640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f5c101090 0x7f2f5c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f53fff640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f5c101090 0x7f2f5c199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f53fff640 1 -- 172.21.15.26:0/3657234909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5c1a4d50 con 0x7f2f5c106160 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f5b59e640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f5c199fd0 0x7f2f5c1a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f53fff640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5c106160 0x7f2f5c199a40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2f44005fd0 tx=0x7f2f4400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.502+0000 7f2f58d99640 1 -- 172.21.15.26:0/3657234909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44017070 con 0x7f2f5c106160 2024-04-21T10:31:07.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.503+0000 7f2f61248640 1 -- 172.21.15.26:0/3657234909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5c1a4fe0 con 0x7f2f5c106160 2024-04-21T10:31:07.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.503+0000 7f2f58d99640 1 -- 172.21.15.26:0/3657234909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f44009c30 con 0x7f2f5c106160 2024-04-21T10:31:07.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.503+0000 7f2f58d99640 1 -- 172.21.15.26:0/3657234909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44003d30 con 0x7f2f5c106160 2024-04-21T10:31:07.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.503+0000 7f2f61248640 1 -- 172.21.15.26:0/3657234909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f5c1a5550 con 0x7f2f5c106160 2024-04-21T10:31:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.505+0000 7f2f58d99640 1 -- 172.21.15.26:0/3657234909 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f2f44003ed0 con 0x7f2f5c106160 2024-04-21T10:31:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.505+0000 7f2f63e30740 1 Processor -- start 2024-04-21T10:31:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.505+0000 7f2f63e30740 1 -- start start 2024-04-21T10:31:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f63e30740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906c57b0 0x55a6906c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f63e30740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c6160 0x55a6906ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f63e30740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 0x55a6906d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f63e30740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a690629b10 con 0x55a6906cece0 2024-04-21T10:31:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f63e30740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a6905c7240 con 0x55a6906c57b0 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f63e30740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a690625ad0 con 0x55a6906c6160 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f58d99640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f2f480a3ed0 0x7f2f480a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f53fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 0x55a6906d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f53fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 0x55a6906d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33334/0 (socket says 172.21.15.26:33334) 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f53fff640 1 -- 172.21.15.26:0/3522046880 learned_addr learned my addr 172.21.15.26:0/3522046880 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f5b59e640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f2f480a3ed0 0x7f2f480a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f5ad9d640 1 --2- 172.21.15.26:0/3522046880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c6160 0x55a6906ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f5b59e640 1 --2- 172.21.15.26:0/3522046880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906c57b0 0x55a6906c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f58d99640 1 -- 172.21.15.26:0/3657234909 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f2f440d08a0 con 0x7f2f5c106160 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f53fff640 1 -- 172.21.15.26:0/3522046880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c6160 msgr2=0x55a6906ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f53fff640 1 --2- 172.21.15.26:0/3522046880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c6160 0x55a6906ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f53fff640 1 -- 172.21.15.26:0/3522046880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906c57b0 msgr2=0x55a6906c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f53fff640 1 --2- 172.21.15.26:0/3522046880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906c57b0 0x55a6906c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f53fff640 1 -- 172.21.15.26:0/3522046880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69033c2d0 con 0x55a6906cece0 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f5b59e640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f2f480a3ed0 0x7f2f480a6390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f540146e0 tx=0x7f2f5400a920 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.506+0000 7f2f53fff640 1 --2- 172.21.15.26:0/3522046880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 0x55a6906d1130 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2f440a4c70 tx=0x7f2f44085a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8acd725d4116d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.507+0000 7f2f517fa640 1 -- 172.21.15.26:0/3522046880 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44009de0 con 0x55a6906cece0 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.507+0000 7f2f517fa640 1 -- 172.21.15.26:0/3522046880 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f4409c530 con 0x55a6906cece0 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.507+0000 7f2f517fa640 1 -- 172.21.15.26:0/3522046880 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4409c990 con 0x55a6906cece0 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.507+0000 7f2f63e30740 1 -- 172.21.15.26:0/3522046880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 msgr2=0x55a6906d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.507+0000 7f2f63e30740 1 --2- 172.21.15.26:0/3522046880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 0x55a6906d1130 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2f440a4c70 tx=0x7f2f44085a70 comp rx=0 tx=0).stop 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.507+0000 7f2f63e30740 1 -- 172.21.15.26:0/3522046880 shutdown_connections 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.507+0000 7f2f63e30740 1 --2- 172.21.15.26:0/3522046880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 0x55a6906d1130 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.507+0000 7f2f63e30740 1 --2- 172.21.15.26:0/3522046880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c6160 0x55a6906ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.507+0000 7f2f63e30740 1 --2- 172.21.15.26:0/3522046880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906c57b0 0x55a6906c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.507+0000 7f2f63e30740 1 -- 172.21.15.26:0/3522046880 >> 172.21.15.26:0/3522046880 conn(0x55a6906bfa60 msgr2=0x55a6906bfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.507+0000 7f2f63e30740 1 -- 172.21.15.26:0/3522046880 shutdown_connections 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.507+0000 7f2f63e30740 1 -- 172.21.15.26:0/3522046880 wait complete. 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f63e30740 1 -- start start 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f63e30740 1 Processor -- start 2024-04-21T10:31:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f63e30740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f63e30740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 0x55a6906cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f63e30740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906cece0 0x55a6906cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f63e30740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6906d1bf0 con 0x55a6906c6160 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f63e30740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a6905b2190 con 0x55a6906cece0 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f63e30740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a6906d1d70 con 0x55a6906c57b0 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f63e30740 10 client.0 _ll_register_callbacks cb 0x55a6905ad8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f53fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 0x55a6906cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f53fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 0x55a6906cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33338/0 (socket says 172.21.15.26:33338) 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f53fff640 1 -- 172.21.15.26:0/508423177 learned_addr learned my addr 172.21.15.26:0/508423177 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f5ad9d640 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f5b59e640 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906cece0 0x55a6906cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f53fff640 1 -- 172.21.15.26:0/508423177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 msgr2=0x55a6906cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.508+0000 7f2f53fff640 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f53fff640 1 -- 172.21.15.26:0/508423177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906cece0 msgr2=0x55a6906cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f53fff640 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906cece0 0x55a6906cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f53fff640 1 -- 172.21.15.26:0/508423177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6904066b0 con 0x55a6906c6160 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f53fff640 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 0x55a6906cc280 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2f44000c00 tx=0x7f2f44098ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ad8fa04cabae37 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f427fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f427fc640 1 -- 172.21.15.26:0/508423177 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f440b4020 con 0x55a6906c6160 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f427fc640 1 -- 172.21.15.26:0/508423177 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f2f4409c990 con 0x55a6906c6160 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f63e30740 1 -- 172.21.15.26:0/508423177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f4409c530 con 0x55a6906c6160 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f427fc640 1 -- 172.21.15.26:0/508423177 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2f440a51a0 con 0x55a6906c6160 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f63e30740 10 client.15558 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f63e30740 1 -- 172.21.15.26:0/508423177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a6906ddf50 con 0x55a6906c6160 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f427fc640 1 -- 172.21.15.26:0/508423177 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f4409c530 con 0x55a6906c6160 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f427fc640 1 -- 172.21.15.26:0/508423177 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2f440bb170 con 0x55a6906c6160 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f63e30740 10 client.15558 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f63e30740 10 client.15558 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f63e30740 20 client.15558 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:07.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f63e30740 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55a6906df2a0 0x55a6906e1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f63e30740 4 client.15558 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.509+0000 7f2f63e30740 1 -- 172.21.15.26:0/508423177 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a690637530 con 0x55a6906df2a0 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.510+0000 7f2f5ad9d640 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55a6906df2a0 0x55a6906e1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.510+0000 7f2f427fc640 10 client.15558 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.510+0000 7f2f5ad9d640 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55a6906df2a0 0x55a6906e1460 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=942ea09db1adab37 server_cookie=7d3ceac15fbea924 in_seq=0 out_seq=0 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f427fc640 1 -- 172.21.15.26:0/508423177 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a690637530 con 0x55a6906df2a0 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f427fc640 10 client.15558 handle_command_reply: tid=0 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f63e30740 1 client.15558 shutdown 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f63e30740 1 -- 172.21.15.26:0/508423177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 msgr2=0x55a6906cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f63e30740 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 0x55a6906cc280 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2f44000c00 tx=0x7f2f44098ba0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f63e30740 1 -- 172.21.15.26:0/508423177 shutdown_connections 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f63e30740 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55a6906df2a0 0x55a6906e1460 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f63e30740 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906cece0 0x55a6906cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f427fc640 0 client.15558 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f63e30740 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 0x55a6906cc280 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f63e30740 1 --2- 172.21.15.26:0/508423177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f63e30740 1 -- 172.21.15.26:0/508423177 >> 172.21.15.26:0/508423177 conn(0x55a6906bfa60 msgr2=0x55a6906c1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f63e30740 1 -- 172.21.15.26:0/508423177 shutdown_connections 2024-04-21T10:31:07.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f63e30740 1 -- 172.21.15.26:0/508423177 wait complete. 2024-04-21T10:31:07.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.511+0000 7f2f63e30740 20 client.15558 trim_cache size 0 max 16384 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.529+0000 7f2f63e30740 1 Processor -- start 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f63e30740 1 -- start start 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f63e30740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f63e30740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 0x55a6906db670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f63e30740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906cece0 0x55a6906bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f63e30740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6906df7a0 con 0x55a6906c6160 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f63e30740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a6906dfe70 con 0x55a6906cece0 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f63e30740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a6906e0540 con 0x55a6906c57b0 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f53fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 0x55a6906db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f53fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 0x55a6906db670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33348/0 (socket says 172.21.15.26:33348) 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f53fff640 1 -- 172.21.15.26:0/3533981171 learned_addr learned my addr 172.21.15.26:0/3533981171 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f5b59e640 1 --2- 172.21.15.26:0/3533981171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906cece0 0x55a6906bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f5ad9d640 1 --2- 172.21.15.26:0/3533981171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f53fff640 1 -- 172.21.15.26:0/3533981171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 msgr2=0x55a6906ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f53fff640 1 --2- 172.21.15.26:0/3533981171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f53fff640 1 -- 172.21.15.26:0/3533981171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906cece0 msgr2=0x55a6906bfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f53fff640 1 --2- 172.21.15.26:0/3533981171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906cece0 0x55a6906bfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f53fff640 1 -- 172.21.15.26:0/3533981171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6906c4a70 con 0x55a6906c6160 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f5b59e640 1 --2- 172.21.15.26:0/3533981171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906cece0 0x55a6906bfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f5ad9d640 1 --2- 172.21.15.26:0/3533981171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.530+0000 7f2f53fff640 1 --2- 172.21.15.26:0/3533981171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 0x55a6906db670 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2f4409c4c0 tx=0x7f2f44085a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4015a65e1a657981 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.531+0000 7f2f50ff9640 1 -- 172.21.15.26:0/3533981171 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f440b4070 con 0x55a6906c6160 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.531+0000 7f2f50ff9640 1 -- 172.21.15.26:0/3533981171 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f44009de0 con 0x55a6906c6160 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.531+0000 7f2f50ff9640 1 -- 172.21.15.26:0/3533981171 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f440041c0 con 0x55a6906c6160 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.531+0000 7f2f63e30740 1 -- 172.21.15.26:0/3533981171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 msgr2=0x55a6906db670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.531+0000 7f2f63e30740 1 --2- 172.21.15.26:0/3533981171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 0x55a6906db670 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2f4409c4c0 tx=0x7f2f44085a70 comp rx=0 tx=0).stop 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.531+0000 7f2f63e30740 1 -- 172.21.15.26:0/3533981171 shutdown_connections 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.531+0000 7f2f63e30740 1 --2- 172.21.15.26:0/3533981171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906cece0 0x55a6906bfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.531+0000 7f2f63e30740 1 --2- 172.21.15.26:0/3533981171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906c6160 0x55a6906db670 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.531+0000 7f2f63e30740 1 --2- 172.21.15.26:0/3533981171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.531+0000 7f2f63e30740 1 -- 172.21.15.26:0/3533981171 >> 172.21.15.26:0/3533981171 conn(0x55a6906bfa60 msgr2=0x55a6906bea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.531+0000 7f2f63e30740 1 -- 172.21.15.26:0/3533981171 shutdown_connections 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.531+0000 7f2f63e30740 1 -- 172.21.15.26:0/3533981171 wait complete. 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f63e30740 1 -- start start 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f63e30740 1 Processor -- start 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f63e30740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f63e30740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906c6160 0x55a6906c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f63e30740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 0x55a6906e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f63e30740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a6906e12e0 con 0x55a6906cece0 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f63e30740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a6906e0c10 con 0x55a6906c6160 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f63e30740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55a6906e19b0 con 0x55a6906c57b0 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f63e30740 10 client.0 _ll_register_callbacks cb 0x7f2f4c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f5b59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 0x55a6906e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f5b59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 0x55a6906e86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33354/0 (socket says 172.21.15.26:33354) 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f5b59e640 1 -- 172.21.15.26:0/2791284725 learned_addr learned my addr 172.21.15.26:0/2791284725 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f53fff640 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906c6160 0x55a6906c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f5ad9d640 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f5b59e640 1 -- 172.21.15.26:0/2791284725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 msgr2=0x55a6906c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f5b59e640 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f5b59e640 1 -- 172.21.15.26:0/2791284725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906c6160 msgr2=0x55a6906c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f5b59e640 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906c6160 0x55a6906c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.532+0000 7f2f5b59e640 1 -- 172.21.15.26:0/2791284725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6906e8bf0 con 0x55a6906cece0 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f5ad9d640 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f5b59e640 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 0x55a6906e86b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2f54011980 tx=0x7f2f54016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781819808c05e413 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f427fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f427fc640 1 -- 172.21.15.26:0/2791284725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54025070 con 0x55a6906cece0 2024-04-21T10:31:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f427fc640 1 -- 172.21.15.26:0/2791284725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f2f54012de0 con 0x55a6906cece0 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f63e30740 1 -- 172.21.15.26:0/2791284725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f44009de0 con 0x55a6906cece0 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f427fc640 1 -- 172.21.15.26:0/2791284725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2f54030050 con 0x55a6906cece0 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f63e30740 10 client.15564 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f63e30740 1 -- 172.21.15.26:0/2791284725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a6906e9e60 con 0x55a6906cece0 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f427fc640 1 -- 172.21.15.26:0/2791284725 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f44009de0 con 0x55a6906cece0 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f427fc640 1 -- 172.21.15.26:0/2791284725 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2f540056b0 con 0x55a6906cece0 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f63e30740 10 client.15564 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f63e30740 10 client.15564 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f63e30740 20 client.15564 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f63e30740 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55a69074a760 0x55a690749470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f63e30740 4 client.15564 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.533+0000 7f2f63e30740 1 -- 172.21.15.26:0/2791284725 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "status"}) -- 0x55a6906313a0 con 0x55a69074a760 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.534+0000 7f2f5ad9d640 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55a69074a760 0x55a690749470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.534+0000 7f2f427fc640 10 client.15564 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.534+0000 7f2f5ad9d640 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55a69074a760 0x55a690749470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dcc622cd35320f server_cookie=71c166ced988b74 in_seq=0 out_seq=0 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f427fc640 1 -- 172.21.15.26:0/2791284725 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a6906313a0 con 0x55a69074a760 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f427fc640 10 client.15564 handle_command_reply: tid=0 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f63e30740 1 client.15564 shutdown 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f63e30740 1 -- 172.21.15.26:0/2791284725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 msgr2=0x55a6906e86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f63e30740 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 0x55a6906e86b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2f54011980 tx=0x7f2f54016040 comp rx=0 tx=0).stop 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f63e30740 1 -- 172.21.15.26:0/2791284725 shutdown_connections 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f63e30740 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55a69074a760 0x55a690749470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f63e30740 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6906cece0 0x55a6906e86b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f63e30740 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6906c6160 0x55a6906c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f427fc640 0 client.15564 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f63e30740 1 --2- 172.21.15.26:0/2791284725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a6906c57b0 0x55a6906c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f63e30740 1 -- 172.21.15.26:0/2791284725 >> 172.21.15.26:0/2791284725 conn(0x55a6906bfa60 msgr2=0x55a6906bea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f63e30740 1 -- 172.21.15.26:0/2791284725 shutdown_connections 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f63e30740 1 -- 172.21.15.26:0/2791284725 wait complete. 2024-04-21T10:31:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f63e30740 20 client.15564 trim_cache size 0 max 16384 2024-04-21T10:31:07.541 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:31:07.541 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "b77245c8-ffc8-11ee-bc93-c7b262605968", 2024-04-21T10:31:07.541 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-21T10:31:07.541 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24496, 2024-04-21T10:31:07.541 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-21T10:31:07.541 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-21T10:31:07.541 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-21T10:31:07.541 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 125.62294668600001, 2024-04-21T10:31:07.541 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 21, 2024-04-21T10:31:07.541 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 90, 2024-04-21T10:31:07.541 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T10:31:07.541 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 139.57110088799999 2024-04-21T10:31:07.542 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:31:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f61248640 1 -- 172.21.15.26:0/3657234909 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f2f480a3ed0 msgr2=0x7f2f480a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f61248640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f2f480a3ed0 0x7f2f480a6390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f540146e0 tx=0x7f2f5400a920 comp rx=0 tx=0).stop 2024-04-21T10:31:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f61248640 1 -- 172.21.15.26:0/3657234909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5c106160 msgr2=0x7f2f5c199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f61248640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5c106160 0x7f2f5c199a40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2f44005fd0 tx=0x7f2f4400dde0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f61248640 1 -- 172.21.15.26:0/3657234909 shutdown_connections 2024-04-21T10:31:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f61248640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f2f480a3ed0 0x7f2f480a6390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f61248640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f5c199fd0 0x7f2f5c1a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f61248640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f5c106160 0x7f2f5c199a40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f61248640 1 --2- 172.21.15.26:0/3657234909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f5c101090 0x7f2f5c199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.535+0000 7f2f61248640 1 -- 172.21.15.26:0/3657234909 >> 172.21.15.26:0/3657234909 conn(0x7f2f5c077f50 msgr2=0x7f2f5c0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.536+0000 7f2f61248640 1 -- 172.21.15.26:0/3657234909 shutdown_connections 2024-04-21T10:31:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:07.536+0000 7f2f61248640 1 -- 172.21.15.26:0/3657234909 wait complete. 2024-04-21T10:31:07.546 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-21T10:31:07.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:07 smithi169 ceph-mon[31469]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 170 KiB/s wr, 103 op/s 2024-04-21T10:31:07.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:07 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/89724914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:07.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:07 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/837684328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:31:07.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:07 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/4197061070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:07.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:07 smithi190 ceph-mon[31423]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 170 KiB/s wr, 103 op/s 2024-04-21T10:31:07.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:07 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/89724914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:07.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:07 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/837684328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:31:07.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:07 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/4197061070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:09.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:09 smithi026 ceph-mon[27965]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-21T10:31:09.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:09 smithi169 ceph-mon[31469]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-21T10:31:09.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:09 smithi190 ceph-mon[31423]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-21T10:31:11.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:11 smithi026 ceph-mon[27965]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-04-21T10:31:11.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:11 smithi169 ceph-mon[31469]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-04-21T10:31:11.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:11 smithi190 ceph-mon[31423]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-04-21T10:31:13.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:13 smithi026 ceph-mon[27965]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-04-21T10:31:13.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:13 smithi169 ceph-mon[31469]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-04-21T10:31:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.759+0000 7fc2e27db640 1 -- 172.21.15.26:0/1519617926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc1010f0 msgr2=0x7fc2dc1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:13.758 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.759+0000 7fc2e27db640 1 --2- 172.21.15.26:0/1519617926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc1010f0 0x7fc2dc1014d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009900 tx=0x7fc2cc02bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.759+0000 7fc2e27db640 1 -- 172.21.15.26:0/1519617926 shutdown_connections 2024-04-21T10:31:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.759+0000 7fc2e27db640 1 --2- 172.21.15.26:0/1519617926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2dc106170 0x7fc2dc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.759+0000 7fc2e27db640 1 --2- 172.21.15.26:0/1519617926 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2dc101aa0 0x7fc2dc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.759+0000 7fc2e27db640 1 --2- 172.21.15.26:0/1519617926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc1010f0 0x7fc2dc1014d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.759+0000 7fc2e27db640 1 -- 172.21.15.26:0/1519617926 >> 172.21.15.26:0/1519617926 conn(0x7fc2dc077fd0 msgr2=0x7fc2dc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.759+0000 7fc2e27db640 1 -- 172.21.15.26:0/1519617926 shutdown_connections 2024-04-21T10:31:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.759+0000 7fc2e27db640 1 -- 172.21.15.26:0/1519617926 wait complete. 2024-04-21T10:31:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.760+0000 7fc2e27db640 1 Processor -- start 2024-04-21T10:31:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.760+0000 7fc2e27db640 1 -- start start 2024-04-21T10:31:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.760+0000 7fc2e27db640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2dc1010f0 0x7fc2dc199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.760+0000 7fc2e27db640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2dc106170 0x7fc2dc199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.760+0000 7fc2e27db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc199fd0 0x7fc2dc1a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.760+0000 7fc2e27db640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc1110b0 con 0x7fc2dc199fd0 2024-04-21T10:31:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.760+0000 7fc2e27db640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc110f30 con 0x7fc2dc106170 2024-04-21T10:31:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.760+0000 7fc2e27db640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc111230 con 0x7fc2dc1010f0 2024-04-21T10:31:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2e0d78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc199fd0 0x7fc2dc1a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2db7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2dc106170 0x7fc2dc199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2e0d78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc199fd0 0x7fc2dc1a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33372/0 (socket says 172.21.15.26:33372) 2024-04-21T10:31:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2e0d78640 1 -- 172.21.15.26:0/1929060674 learned_addr learned my addr 172.21.15.26:0/1929060674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2dbfff640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2dc1010f0 0x7fc2dc199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2e0d78640 1 -- 172.21.15.26:0/1929060674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2dc1010f0 msgr2=0x7fc2dc199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2e0d78640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2dc1010f0 0x7fc2dc199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2e0d78640 1 -- 172.21.15.26:0/1929060674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2dc106170 msgr2=0x7fc2dc199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2e0d78640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2dc106170 0x7fc2dc199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2e0d78640 1 -- 172.21.15.26:0/1929060674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2dc1a4d50 con 0x7fc2dc199fd0 2024-04-21T10:31:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2e0d78640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc199fd0 0x7fc2dc1a45d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400cce0 tx=0x7fc2d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2d97fa640 1 -- 172.21.15.26:0/1929060674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d400f450 con 0x7fc2dc199fd0 2024-04-21T10:31:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2d97fa640 1 -- 172.21.15.26:0/1929060674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d400faa0 con 0x7fc2dc199fd0 2024-04-21T10:31:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2e27db640 1 -- 172.21.15.26:0/1929060674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2dc1a5040 con 0x7fc2dc199fd0 2024-04-21T10:31:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2d97fa640 1 -- 172.21.15.26:0/1929060674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4018b70 con 0x7fc2dc199fd0 2024-04-21T10:31:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.761+0000 7fc2e27db640 1 -- 172.21.15.26:0/1929060674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2dc1a5560 con 0x7fc2dc199fd0 2024-04-21T10:31:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.764+0000 7fc2e53ce740 1 Processor -- start 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.764+0000 7fc2d97fa640 1 -- 172.21.15.26:0/1929060674 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fc2d4018d10 con 0x7fc2dc199fd0 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2e53ce740 1 -- start start 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2e53ce740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf11628d0 0x556bf1161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2e53ce740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf1161b60 0x556bf1161fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2e53ce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 0x556bf116d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2e53ce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556bf10c68d0 con 0x556bf116b0e0 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2e53ce740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556bf1063e80 con 0x556bf1161b60 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2e53ce740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556bf10c2890 con 0x556bf11628d0 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 0x556bf116d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 0x556bf116d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33378/0 (socket says 172.21.15.26:33378) 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2db7fe640 1 -- 172.21.15.26:0/1779003814 learned_addr learned my addr 172.21.15.26:0/1779003814 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2dbfff640 1 --2- 172.21.15.26:0/1779003814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf1161b60 0x556bf1161fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.766 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/1779003814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf11628d0 0x556bf1161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2d97fa640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc2c40a3ff0 0x7fc2c40a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2db7fe640 1 -- 172.21.15.26:0/1779003814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf11628d0 msgr2=0x556bf1161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/1779003814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf11628d0 0x556bf1161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2db7fe640 1 -- 172.21.15.26:0/1779003814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf1161b60 msgr2=0x556bf1161fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/1779003814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf1161b60 0x556bf1161fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2db7fe640 1 -- 172.21.15.26:0/1779003814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bf1085320 con 0x556bf116b0e0 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.765+0000 7fc2e0d78640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc2c40a3ff0 0x7fc2c40a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.766+0000 7fc2d97fa640 1 -- 172.21.15.26:0/1929060674 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fc2d40d0db0 con 0x7fc2dc199fd0 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.766+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/1779003814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 0x556bf116d4d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0012580 tx=0x7fc2d000b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368abfb644368980 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.766+0000 7fc2c27fc640 1 -- 172.21.15.26:0/1779003814 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d001e640 con 0x556bf116b0e0 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.766+0000 7fc2c27fc640 1 -- 172.21.15.26:0/1779003814 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d00108e0 con 0x556bf116b0e0 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.766+0000 7fc2c27fc640 1 -- 172.21.15.26:0/1779003814 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d001d940 con 0x556bf116b0e0 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.767+0000 7fc2e53ce740 1 -- 172.21.15.26:0/1779003814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 msgr2=0x556bf116d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.767+0000 7fc2e0d78640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc2c40a3ff0 0x7fc2c40a64b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40ce330 tx=0x7fc2d4067670 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:13.767 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.767+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/1779003814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 0x556bf116d4d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0012580 tx=0x7fc2d000b9d0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.767+0000 7fc2e53ce740 1 -- 172.21.15.26:0/1779003814 shutdown_connections 2024-04-21T10:31:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.767+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/1779003814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 0x556bf116d4d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.767+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/1779003814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf1161b60 0x556bf1161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.767+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/1779003814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf11628d0 0x556bf1161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.767+0000 7fc2e53ce740 1 -- 172.21.15.26:0/1779003814 >> 172.21.15.26:0/1779003814 conn(0x556bf1128c90 msgr2=0x556bf115c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.767+0000 7fc2e53ce740 1 -- 172.21.15.26:0/1779003814 shutdown_connections 2024-04-21T10:31:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.767+0000 7fc2e53ce740 1 -- 172.21.15.26:0/1779003814 wait complete. 2024-04-21T10:31:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.767+0000 7fc2e53ce740 1 -- start start 2024-04-21T10:31:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2e53ce740 1 Processor -- start 2024-04-21T10:31:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2e53ce740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf1161b60 0x556bf116aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2e53ce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf11628d0 0x556bf1167e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2e53ce740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf116b0e0 0x556bf1168480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2e53ce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556bf116de10 con 0x556bf11628d0 2024-04-21T10:31:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2e53ce740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556bf104eeb0 con 0x556bf1161b60 2024-04-21T10:31:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2e53ce740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x556bf116df90 con 0x556bf116b0e0 2024-04-21T10:31:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2e53ce740 10 client.0 _ll_register_callbacks cb 0x556bf107bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf11628d0 0x556bf1167e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf11628d0 0x556bf1167e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33380/0 (socket says 172.21.15.26:33380) 2024-04-21T10:31:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2dbfff640 1 -- 172.21.15.26:0/3858699557 learned_addr learned my addr 172.21.15.26:0/3858699557 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf1161b60 0x556bf116aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf116b0e0 0x556bf1168480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2dbfff640 1 -- 172.21.15.26:0/3858699557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf116b0e0 msgr2=0x556bf1168480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2dbfff640 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf116b0e0 0x556bf1168480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2dbfff640 1 -- 172.21.15.26:0/3858699557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf1161b60 msgr2=0x556bf116aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2dbfff640 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf1161b60 0x556bf116aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2dbfff640 1 -- 172.21.15.26:0/3858699557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bf0dd9210 con 0x556bf11628d0 2024-04-21T10:31:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.768+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf116b0e0 0x556bf1168480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2dbfff640 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf11628d0 0x556bf1167e70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009900 tx=0x7fc2cc02c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98771fb307aa2aad server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2bb7fe640 1 -- 172.21.15.26:0/3858699557 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc005030 con 0x556bf11628d0 2024-04-21T10:31:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2bb7fe640 1 -- 172.21.15.26:0/3858699557 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fc2cc0051d0 con 0x556bf11628d0 2024-04-21T10:31:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3858699557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2d00108e0 con 0x556bf11628d0 2024-04-21T10:31:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2bb7fe640 1 -- 172.21.15.26:0/3858699557 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc2cc0053d0 con 0x556bf11628d0 2024-04-21T10:31:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2e53ce740 10 client.15576 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3858699557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556bf0ea5620 con 0x556bf11628d0 2024-04-21T10:31:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2bb7fe640 1 -- 172.21.15.26:0/3858699557 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d00108e0 con 0x556bf11628d0 2024-04-21T10:31:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2bb7fe640 1 -- 172.21.15.26:0/3858699557 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc2cc003860 con 0x556bf11628d0 2024-04-21T10:31:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2e53ce740 10 client.15576 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2e53ce740 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:31:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2e53ce740 20 client.15576 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x556bf117b0e0 0x556bf117d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.769+0000 7fc2e53ce740 4 client.15576 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.770+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3858699557 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bf10da880 con 0x556bf117b0e0 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.770+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x556bf117b0e0 0x556bf117d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.770+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x556bf117b0e0 0x556bf117d530 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39cea058f1c8a352 server_cookie=c386595f4494cb82 in_seq=0 out_seq=0 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.770+0000 7fc2bb7fe640 10 client.15576 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2bb7fe640 1 -- 172.21.15.26:0/3858699557 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556bf10da880 con 0x556bf117b0e0 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2bb7fe640 10 client.15576 handle_command_reply: tid=0 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2e53ce740 1 client.15576 shutdown 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3858699557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf11628d0 msgr2=0x556bf1167e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf11628d0 0x556bf1167e70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009900 tx=0x7fc2cc02c9f0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3858699557 shutdown_connections 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x556bf117b0e0 0x556bf117d530 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf116b0e0 0x556bf1168480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf11628d0 0x556bf1167e70 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/3858699557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf1161b60 0x556bf116aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3858699557 >> 172.21.15.26:0/3858699557 conn(0x556bf1128c90 msgr2=0x556bf115e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2bb7fe640 0 client.15576 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3858699557 shutdown_connections 2024-04-21T10:31:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.771+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3858699557 wait complete. 2024-04-21T10:31:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.772+0000 7fc2e53ce740 20 client.15576 trim_cache size 0 max 16384 2024-04-21T10:31:13.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:13 smithi190 ceph-mon[31423]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-04-21T10:31:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.787+0000 7fc2e53ce740 1 Processor -- start 2024-04-21T10:31:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.787+0000 7fc2e53ce740 1 -- start start 2024-04-21T10:31:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2e53ce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf1161b60 0x556bf1176d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2e53ce740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf11628d0 0x556bf116a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2e53ce740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf116b0e0 0x556bf116aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2e53ce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc1b0340 con 0x556bf1161b60 2024-04-21T10:31:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2e53ce740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc1adc10 con 0x556bf116b0e0 2024-04-21T10:31:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2e53ce740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc1aba80 con 0x556bf11628d0 2024-04-21T10:31:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf1161b60 0x556bf1176d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf1161b60 0x556bf1176d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33382/0 (socket says 172.21.15.26:33382) 2024-04-21T10:31:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2db7fe640 1 -- 172.21.15.26:0/1999590373 learned_addr learned my addr 172.21.15.26:0/1999590373 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2dbfff640 1 --2- 172.21.15.26:0/1999590373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf11628d0 0x556bf116a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2db7fe640 1 -- 172.21.15.26:0/1999590373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf11628d0 msgr2=0x556bf116a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/1999590373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf11628d0 0x556bf116a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2db7fe640 1 -- 172.21.15.26:0/1999590373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf116b0e0 msgr2=0x556bf116aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/1999590373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf116b0e0 0x556bf116aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2db7fe640 1 -- 172.21.15.26:0/1999590373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bf1160e30 con 0x556bf1161b60 2024-04-21T10:31:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2dbfff640 1 --2- 172.21.15.26:0/1999590373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf11628d0 0x556bf116a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/1999590373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf1161b60 0x556bf1176d10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc2d001d450 tx=0x7fc2d00123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be35e1348dfc8e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2c1ffb640 1 -- 172.21.15.26:0/1999590373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0023040 con 0x556bf1161b60 2024-04-21T10:31:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.788+0000 7fc2c1ffb640 1 -- 172.21.15.26:0/1999590373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d001d650 con 0x556bf1161b60 2024-04-21T10:31:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.789+0000 7fc2c1ffb640 1 -- 172.21.15.26:0/1999590373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d002b3a0 con 0x556bf1161b60 2024-04-21T10:31:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.789+0000 7fc2e53ce740 1 -- 172.21.15.26:0/1999590373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf1161b60 msgr2=0x556bf1176d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.789+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/1999590373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf1161b60 0x556bf1176d10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc2d001d450 tx=0x7fc2d00123f0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.789+0000 7fc2e53ce740 1 -- 172.21.15.26:0/1999590373 shutdown_connections 2024-04-21T10:31:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.789+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/1999590373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf116b0e0 0x556bf116aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.789+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/1999590373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf11628d0 0x556bf116a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.789+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/1999590373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf1161b60 0x556bf1176d10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.789+0000 7fc2e53ce740 1 -- 172.21.15.26:0/1999590373 >> 172.21.15.26:0/1999590373 conn(0x556bf1128c90 msgr2=0x556bf1157760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.789+0000 7fc2e53ce740 1 -- 172.21.15.26:0/1999590373 shutdown_connections 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.789+0000 7fc2e53ce740 1 -- 172.21.15.26:0/1999590373 wait complete. 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2e53ce740 1 -- start start 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2e53ce740 1 Processor -- start 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2e53ce740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf1161b60 0x556bf116cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2e53ce740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf11628d0 0x556bf1173220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2e53ce740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 0x556bf11650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2e53ce740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc1ac0d0 con 0x556bf116b0e0 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2e53ce740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc1ad540 con 0x556bf11628d0 2024-04-21T10:31:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2e53ce740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc1ac7a0 con 0x556bf1161b60 2024-04-21T10:31:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2e53ce740 10 client.0 _ll_register_callbacks cb 0x7fc2cc02c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 0x556bf11650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 0x556bf11650f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33390/0 (socket says 172.21.15.26:33390) 2024-04-21T10:31:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2db7fe640 1 -- 172.21.15.26:0/3826677786 learned_addr learned my addr 172.21.15.26:0/3826677786 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.790+0000 7fc2dbfff640 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf11628d0 0x556bf1173220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf1161b60 0x556bf116cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.796 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2db7fe640 1 -- 172.21.15.26:0/3826677786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf1161b60 msgr2=0x556bf116cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf1161b60 0x556bf116cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2db7fe640 1 -- 172.21.15.26:0/3826677786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf11628d0 msgr2=0x556bf1173220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf11628d0 0x556bf1173220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2db7fe640 1 -- 172.21.15.26:0/3826677786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bf1165630 con 0x556bf116b0e0 2024-04-21T10:31:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2db7fe640 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 0x556bf11650f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00051f0 tx=0x7fc2d0012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccefbf63ec66525 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2bb7fe640 1 -- 172.21.15.26:0/3826677786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0030040 con 0x556bf116b0e0 2024-04-21T10:31:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2bb7fe640 1 -- 172.21.15.26:0/3826677786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fc2d00053d0 con 0x556bf116b0e0 2024-04-21T10:31:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3826677786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2d001d650 con 0x556bf116b0e0 2024-04-21T10:31:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2bb7fe640 1 -- 172.21.15.26:0/3826677786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc2d00055d0 con 0x556bf116b0e0 2024-04-21T10:31:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2e53ce740 10 client.15582 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3826677786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556bf116bae0 con 0x556bf116b0e0 2024-04-21T10:31:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.791+0000 7fc2bb7fe640 1 -- 172.21.15.26:0/3826677786 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d001d650 con 0x556bf116b0e0 2024-04-21T10:31:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.792+0000 7fc2bb7fe640 1 -- 172.21.15.26:0/3826677786 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc2d001c070 con 0x556bf116b0e0 2024-04-21T10:31:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.792+0000 7fc2e53ce740 10 client.15582 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.792+0000 7fc2e53ce740 10 client.15582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:31:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.792+0000 7fc2e53ce740 20 client.15582 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.792+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x556bf116c000 0x556bf11e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.792+0000 7fc2e53ce740 4 client.15582 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T10:31:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.792+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3826677786 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556bf10663d0 con 0x556bf116c000 2024-04-21T10:31:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.792+0000 7fc2dbfff640 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x556bf116c000 0x556bf11e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.792+0000 7fc2dbfff640 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x556bf116c000 0x556bf11e7ad0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3bb68abbabfbfce server_cookie=7b2a4ae83e90674a in_seq=0 out_seq=0 2024-04-21T10:31:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.792+0000 7fc2bb7fe640 10 client.15582 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2bb7fe640 1 -- 172.21.15.26:0/3826677786 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556bf10663d0 con 0x556bf116c000 2024-04-21T10:31:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2bb7fe640 10 client.15582 handle_command_reply: tid=0 2024-04-21T10:31:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2e53ce740 1 client.15582 shutdown 2024-04-21T10:31:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3826677786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 msgr2=0x556bf11650f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 0x556bf11650f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00051f0 tx=0x7fc2d0012ed0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3826677786 shutdown_connections 2024-04-21T10:31:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x556bf116c000 0x556bf11e7ad0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bf116b0e0 0x556bf11650f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556bf11628d0 0x556bf1173220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2e53ce740 1 --2- 172.21.15.26:0/3826677786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556bf1161b60 0x556bf116cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2bb7fe640 0 client.15582 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3826677786 >> 172.21.15.26:0/3826677786 conn(0x556bf1128c90 msgr2=0x556bf115c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3826677786 shutdown_connections 2024-04-21T10:31:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2e53ce740 1 -- 172.21.15.26:0/3826677786 wait complete. 2024-04-21T10:31:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.804+0000 7fc2e53ce740 20 client.15582 trim_cache size 0 max 16384 2024-04-21T10:31:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-04-21T10:31:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-04-21T10:31:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-04-21T10:31:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-04-21T10:31:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.805+0000 7fc2e27db640 1 -- 172.21.15.26:0/1929060674 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc2c40a3ff0 msgr2=0x7fc2c40a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.805+0000 7fc2e27db640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc2c40a3ff0 0x7fc2c40a64b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40ce330 tx=0x7fc2d4067670 comp rx=0 tx=0).stop 2024-04-21T10:31:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.805+0000 7fc2e27db640 1 -- 172.21.15.26:0/1929060674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc199fd0 msgr2=0x7fc2dc1a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.805+0000 7fc2e27db640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc199fd0 0x7fc2dc1a45d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400cce0 tx=0x7fc2d400a620 comp rx=0 tx=0).stop 2024-04-21T10:31:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.805+0000 7fc2e27db640 1 -- 172.21.15.26:0/1929060674 shutdown_connections 2024-04-21T10:31:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.805+0000 7fc2e27db640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc2c40a3ff0 0x7fc2c40a64b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.805+0000 7fc2e27db640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2dc199fd0 0x7fc2dc1a45d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.805+0000 7fc2e27db640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2dc106170 0x7fc2dc199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.805+0000 7fc2e27db640 1 --2- 172.21.15.26:0/1929060674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2dc1010f0 0x7fc2dc199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.805+0000 7fc2e27db640 1 -- 172.21.15.26:0/1929060674 >> 172.21.15.26:0/1929060674 conn(0x7fc2dc077fd0 msgr2=0x7fc2dc0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.805+0000 7fc2e27db640 1 -- 172.21.15.26:0/1929060674 shutdown_connections 2024-04-21T10:31:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:13.805+0000 7fc2e27db640 1 -- 172.21.15.26:0/1929060674 wait complete. 2024-04-21T10:31:15.578 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:15 smithi190 ceph-mon[31423]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 64 op/s 2024-04-21T10:31:15.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:15 smithi026 ceph-mon[27965]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 64 op/s 2024-04-21T10:31:15.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:15 smithi169 ceph-mon[31469]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 64 op/s 2024-04-21T10:31:17.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:17 smithi026 ceph-mon[27965]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 101 KiB/s wr, 69 op/s 2024-04-21T10:31:17.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:17 smithi169 ceph-mon[31469]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 101 KiB/s wr, 69 op/s 2024-04-21T10:31:17.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:17 smithi190 ceph-mon[31423]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 101 KiB/s wr, 69 op/s 2024-04-21T10:31:19.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:19 smithi026 ceph-mon[27965]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-04-21T10:31:19.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:19 smithi169 ceph-mon[31469]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-04-21T10:31:19.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:19 smithi190 ceph-mon[31423]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-04-21T10:31:21.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:21 smithi026 ceph-mon[27965]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-21T10:31:21.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:21 smithi169 ceph-mon[31469]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-21T10:31:21.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:21 smithi190 ceph-mon[31423]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-21T10:31:22.547 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-21T10:31:22.547 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.95 2024-04-21T10:31:22.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:31:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.653+0000 7f32ccb04640 1 -- 172.21.15.26:0/2694036342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c80690e0 msgr2=0x7f32c8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.653+0000 7f32ccb04640 1 --2- 172.21.15.26:0/2694036342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c80690e0 0x7f32c8069560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009900 tx=0x7f32bc02bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.654+0000 7f32ccb04640 1 -- 172.21.15.26:0/2694036342 shutdown_connections 2024-04-21T10:31:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.654+0000 7f32ccb04640 1 --2- 172.21.15.26:0/2694036342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c8069aa0 0x7f32c810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.654+0000 7f32ccb04640 1 --2- 172.21.15.26:0/2694036342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c80690e0 0x7f32c8069560 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.654+0000 7f32ccb04640 1 --2- 172.21.15.26:0/2694036342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32c8106f50 0x7f32c8107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.654+0000 7f32ccb04640 1 -- 172.21.15.26:0/2694036342 >> 172.21.15.26:0/2694036342 conn(0x7f32c8077e30 msgr2=0x7f32c8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.654+0000 7f32ccb04640 1 -- 172.21.15.26:0/2694036342 shutdown_connections 2024-04-21T10:31:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.654+0000 7f32ccb04640 1 -- 172.21.15.26:0/2694036342 wait complete. 2024-04-21T10:31:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.654+0000 7f32ccb04640 1 Processor -- start 2024-04-21T10:31:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.654+0000 7f32ccb04640 1 -- start start 2024-04-21T10:31:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32ccb04640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32c80690e0 0x7f32c8101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32ccb04640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c8069aa0 0x7f32c81017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32ccb04640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c8106f50 0x7f32c8101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32ccb04640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32c8112170 con 0x7f32c8069aa0 2024-04-21T10:31:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32ccb04640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32c8111ff0 con 0x7f32c80690e0 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32ccb04640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81122f0 con 0x7f32c8106f50 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32c5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c8069aa0 0x7f32c81017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32c5d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c8069aa0 0x7f32c81017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41140/0 (socket says 172.21.15.26:41140) 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32c5d9b640 1 -- 172.21.15.26:0/4107709893 learned_addr learned my addr 172.21.15.26:0/4107709893 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32c659c640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32c80690e0 0x7f32c8101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32c6d9d640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c8106f50 0x7f32c8101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32c5d9b640 1 -- 172.21.15.26:0/4107709893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c8106f50 msgr2=0x7f32c8101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32c5d9b640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c8106f50 0x7f32c8101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32c5d9b640 1 -- 172.21.15.26:0/4107709893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32c80690e0 msgr2=0x7f32c8101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32c5d9b640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32c80690e0 0x7f32c8101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32c5d9b640 1 -- 172.21.15.26:0/4107709893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c81a6b90 con 0x7f32c8069aa0 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32c659c640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32c80690e0 0x7f32c8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.655+0000 7f32c5d9b640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c8069aa0 0x7f32c81017b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f32bc006010 tx=0x7f32bc005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.656+0000 7f32b77fe640 1 -- 172.21.15.26:0/4107709893 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc043020 con 0x7f32c8069aa0 2024-04-21T10:31:22.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.656+0000 7f32ccb04640 1 -- 172.21.15.26:0/4107709893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c81a6e20 con 0x7f32c8069aa0 2024-04-21T10:31:22.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.656+0000 7f32b77fe640 1 -- 172.21.15.26:0/4107709893 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32bc0043d0 con 0x7f32c8069aa0 2024-04-21T10:31:22.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.656+0000 7f32b77fe640 1 -- 172.21.15.26:0/4107709893 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc003b20 con 0x7f32c8069aa0 2024-04-21T10:31:22.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.656+0000 7f32ccb04640 1 -- 172.21.15.26:0/4107709893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c81a7330 con 0x7f32c8069aa0 2024-04-21T10:31:22.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.657+0000 7f32ccb04640 1 -- 172.21.15.26:0/4107709893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c8068de0 con 0x7f32c8069aa0 2024-04-21T10:31:22.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.658+0000 7f32b77fe640 1 -- 172.21.15.26:0/4107709893 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f32bc046020 con 0x7f32c8069aa0 2024-04-21T10:31:22.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.659+0000 7f32b77fe640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f32b00a3ed0 0x7f32b00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:22.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.659+0000 7f32c659c640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f32b00a3ed0 0x7f32b00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:22.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.659+0000 7f32b77fe640 1 -- 172.21.15.26:0/4107709893 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f32bc0f2530 con 0x7f32c8069aa0 2024-04-21T10:31:22.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.660+0000 7f32c659c640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f32b00a3ed0 0x7f32b00a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009f40 tx=0x7f32b8005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:22.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.664+0000 7f32b77fe640 1 -- 172.21.15.26:0/4107709893 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32bc04d070 con 0x7f32c8069aa0 2024-04-21T10:31:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.917+0000 7f32ccb04640 1 -- 172.21.15.26:0/4107709893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32c8109f10 con 0x7f32c8069aa0 2024-04-21T10:31:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.918+0000 7f32b77fe640 1 -- 172.21.15.26:0/4107709893 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7f32bc0beab0 con 0x7f32c8069aa0 2024-04-21T10:31:22.918 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:22.918 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:30:56.186598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24478,24488]},"id":1}]} 2024-04-21T10:31:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 21 2024-04-21T10:31:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.920+0000 7f32ccb04640 1 -- 172.21.15.26:0/4107709893 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f32b00a3ed0 msgr2=0x7f32b00a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.920+0000 7f32ccb04640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f32b00a3ed0 0x7f32b00a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009f40 tx=0x7f32b8005f90 comp rx=0 tx=0).stop 2024-04-21T10:31:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.920+0000 7f32ccb04640 1 -- 172.21.15.26:0/4107709893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c8069aa0 msgr2=0x7f32c81017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.920+0000 7f32ccb04640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c8069aa0 0x7f32c81017b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f32bc006010 tx=0x7f32bc005e20 comp rx=0 tx=0).stop 2024-04-21T10:31:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.920+0000 7f32ccb04640 1 -- 172.21.15.26:0/4107709893 shutdown_connections 2024-04-21T10:31:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.920+0000 7f32ccb04640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f32b00a3ed0 0x7f32b00a6390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.920+0000 7f32ccb04640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32c8106f50 0x7f32c8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.920+0000 7f32ccb04640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c8069aa0 0x7f32c81017b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.920+0000 7f32ccb04640 1 --2- 172.21.15.26:0/4107709893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32c80690e0 0x7f32c8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.920+0000 7f32ccb04640 1 -- 172.21.15.26:0/4107709893 >> 172.21.15.26:0/4107709893 conn(0x7f32c8077e30 msgr2=0x7f32c80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.920+0000 7f32ccb04640 1 -- 172.21.15.26:0/4107709893 shutdown_connections 2024-04-21T10:31:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:22.920+0000 7f32ccb04640 1 -- 172.21.15.26:0/4107709893 wait complete. 2024-04-21T10:31:22.931 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 30 --expiration 78.95 --await' 2024-04-21T10:31:22.931 DEBUG:teuthology.orchestra.run.smithi026:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24496 quiesce db / --timeout 30 --expiration 78.95 --await 2024-04-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.048+0000 7fc785a81640 1 -- 172.21.15.26:0/4051611331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101110 msgr2=0x7fc7801014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.048+0000 7fc785a81640 1 --2- 172.21.15.26:0/4051611331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101110 0x7fc7801014f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc770009900 tx=0x7fc77002bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.048+0000 7fc785a81640 1 -- 172.21.15.26:0/4051611331 shutdown_connections 2024-04-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.048+0000 7fc785a81640 1 --2- 172.21.15.26:0/4051611331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc780106190 0x7fc78010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.048+0000 7fc785a81640 1 --2- 172.21.15.26:0/4051611331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc780101ac0 0x7fc780105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.048+0000 7fc785a81640 1 --2- 172.21.15.26:0/4051611331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101110 0x7fc7801014f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.048+0000 7fc785a81640 1 -- 172.21.15.26:0/4051611331 >> 172.21.15.26:0/4051611331 conn(0x7fc780077fb0 msgr2=0x7fc7800783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.048+0000 7fc785a81640 1 -- 172.21.15.26:0/4051611331 shutdown_connections 2024-04-21T10:31:23.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.048+0000 7fc785a81640 1 -- 172.21.15.26:0/4051611331 wait complete. 2024-04-21T10:31:23.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.049+0000 7fc785a81640 1 Processor -- start 2024-04-21T10:31:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.049+0000 7fc785a81640 1 -- start start 2024-04-21T10:31:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.049+0000 7fc785a81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101110 0x7fc780102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.049+0000 7fc785a81640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc780106190 0x7fc780103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.049+0000 7fc785a81640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7801035d0 0x7fc7801a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.049+0000 7fc785a81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801110d0 con 0x7fc780101110 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.049+0000 7fc785a81640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc780110f50 con 0x7fc780106190 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.049+0000 7fc785a81640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc780111250 con 0x7fc7801035d0 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.049+0000 7fc77f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101110 0x7fc780102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.049+0000 7fc77f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101110 0x7fc780102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41170/0 (socket says 172.21.15.26:41170) 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.049+0000 7fc77f7fe640 1 -- 172.21.15.26:0/1268793269 learned_addr learned my addr 172.21.15.26:0/1268793269 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.049+0000 7fc77effd640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc780106190 0x7fc780103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.050+0000 7fc77ffff640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7801035d0 0x7fc7801a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.050+0000 7fc77f7fe640 1 -- 172.21.15.26:0/1268793269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7801035d0 msgr2=0x7fc7801a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.050+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7801035d0 0x7fc7801a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.050+0000 7fc77f7fe640 1 -- 172.21.15.26:0/1268793269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc780106190 msgr2=0x7fc780103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.050+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc780106190 0x7fc780103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.050+0000 7fc77f7fe640 1 -- 172.21.15.26:0/1268793269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7801a7050 con 0x7fc780101110 2024-04-21T10:31:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.050+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101110 0x7fc780102b50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc770005940 tx=0x7fc7700045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:23.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.050+0000 7fc77cff9640 1 -- 172.21.15.26:0/1268793269 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770043020 con 0x7fc780101110 2024-04-21T10:31:23.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.050+0000 7fc785a81640 1 -- 172.21.15.26:0/1268793269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7801a72e0 con 0x7fc780101110 2024-04-21T10:31:23.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.050+0000 7fc77cff9640 1 -- 172.21.15.26:0/1268793269 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc77002caf0 con 0x7fc780101110 2024-04-21T10:31:23.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.050+0000 7fc77cff9640 1 -- 172.21.15.26:0/1268793269 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770003af0 con 0x7fc780101110 2024-04-21T10:31:23.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.050+0000 7fc785a81640 1 -- 172.21.15.26:0/1268793269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7801a7880 con 0x7fc780101110 2024-04-21T10:31:23.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.053+0000 7fc77cff9640 1 -- 172.21.15.26:0/1268793269 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fc770046020 con 0x7fc780101110 2024-04-21T10:31:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.053+0000 7fc77cff9640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc7680a3f00 0x7fc7680a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.053+0000 7fc788678740 1 Processor -- start 2024-04-21T10:31:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.053+0000 7fc788678740 1 -- start start 2024-04-21T10:31:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77effd640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc7680a3f00 0x7fc7680a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77cff9640 1 -- 172.21.15.26:0/1268793269 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fc7700f2b60 con 0x7fc780101110 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc788678740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597a9c0 0x560a5597ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc788678740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a55983980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc788678740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a55983ec0 0x560a559862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc788678740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560a558db0b0 con 0x560a55983ec0 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc788678740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560a5588c680 con 0x560a5597a9c0 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc788678740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560a55898c60 con 0x560a5597b2e0 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597a9c0 0x560a5597ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a55983ec0 0x560a559862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597a9c0 0x560a5597ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.26:40934/0 (socket says 172.21.15.26:40934) 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77f7fe640 1 -- 172.21.15.26:0/232596178 learned_addr learned my addr 172.21.15.26:0/232596178 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77effd640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc7680a3f00 0x7fc7680a63c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc7801045e0 tx=0x7fc77400b390 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77effd640 1 --2- 172.21.15.26:0/232596178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a55983980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77ffff640 1 -- 172.21.15.26:0/232596178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 msgr2=0x560a55983980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:23.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77ffff640 1 --2- 172.21.15.26:0/232596178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a55983980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77ffff640 1 -- 172.21.15.26:0/232596178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597a9c0 msgr2=0x560a5597ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77ffff640 1 --2- 172.21.15.26:0/232596178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597a9c0 0x560a5597ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77ffff640 1 -- 172.21.15.26:0/232596178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a556bdc90 con 0x560a55983ec0 2024-04-21T10:31:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/232596178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597a9c0 0x560a5597ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.054+0000 7fc77effd640 1 --2- 172.21.15.26:0/232596178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a55983980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.055+0000 7fc77ffff640 1 --2- 172.21.15.26:0/232596178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a55983ec0 0x560a559862b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc778009e00 tx=0x7fc7780117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3400007e884198b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.055+0000 7fc765ffb640 1 -- 172.21.15.26:0/232596178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778003b70 con 0x560a55983ec0 2024-04-21T10:31:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.055+0000 7fc765ffb640 1 -- 172.21.15.26:0/232596178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc778003d10 con 0x560a55983ec0 2024-04-21T10:31:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.055+0000 7fc765ffb640 1 -- 172.21.15.26:0/232596178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778012890 con 0x560a55983ec0 2024-04-21T10:31:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.055+0000 7fc788678740 1 -- 172.21.15.26:0/232596178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a55983ec0 msgr2=0x560a559862b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.055+0000 7fc788678740 1 --2- 172.21.15.26:0/232596178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a55983ec0 0x560a559862b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc778009e00 tx=0x7fc7780117b0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.056+0000 7fc788678740 1 -- 172.21.15.26:0/232596178 shutdown_connections 2024-04-21T10:31:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.056+0000 7fc788678740 1 --2- 172.21.15.26:0/232596178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a55983ec0 0x560a559862b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.056+0000 7fc788678740 1 --2- 172.21.15.26:0/232596178 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a55983980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.056+0000 7fc788678740 1 --2- 172.21.15.26:0/232596178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597a9c0 0x560a5597ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.056+0000 7fc788678740 1 -- 172.21.15.26:0/232596178 >> 172.21.15.26:0/232596178 conn(0x560a559750f0 msgr2=0x560a559754d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.056+0000 7fc788678740 1 -- 172.21.15.26:0/232596178 shutdown_connections 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.056+0000 7fc788678740 1 -- 172.21.15.26:0/232596178 wait complete. 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.056+0000 7fc788678740 1 -- start start 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.056+0000 7fc788678740 1 Processor -- start 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.056+0000 7fc788678740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a55980eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.056+0000 7fc788678740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a559813f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc788678740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a55983ec0 0x560a55981930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc788678740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560a558de400 con 0x560a5597a9c0 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc788678740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560a5589a840 con 0x560a55983ec0 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc788678740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x560a558e3720 con 0x560a5597b2e0 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc788678740 10 client.0 _ll_register_callbacks cb 0x560a558ebea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc77f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a55980eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc77f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a55980eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41184/0 (socket says 172.21.15.26:41184) 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc77f7fe640 1 -- 172.21.15.26:0/1081397083 learned_addr learned my addr 172.21.15.26:0/1081397083 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc77ffff640 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a55983ec0 0x560a55981930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc77effd640 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a559813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc77f7fe640 1 -- 172.21.15.26:0/1081397083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 msgr2=0x560a559813f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a559813f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc77f7fe640 1 -- 172.21.15.26:0/1081397083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a55983ec0 msgr2=0x560a55981930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a55983ec0 0x560a55981930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc77f7fe640 1 -- 172.21.15.26:0/1081397083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5571e420 con 0x560a5597a9c0 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a55980eb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc7700f02d0 tx=0x7fc7700be5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939edc3af6a03d05 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc75effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc75effd640 1 -- 172.21.15.26:0/1081397083 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700be9e0 con 0x560a5597a9c0 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc75effd640 1 -- 172.21.15.26:0/1081397083 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fc7700c71c0 con 0x560a5597a9c0 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.057+0000 7fc788678740 1 -- 172.21.15.26:0/1081397083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc778003d10 con 0x560a5597a9c0 2024-04-21T10:31:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.058+0000 7fc75effd640 1 -- 172.21.15.26:0/1081397083 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc7700c2d90 con 0x560a5597a9c0 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.058+0000 7fc788678740 10 client.15600 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.058+0000 7fc788678740 1 -- 172.21.15.26:0/1081397083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560a555b1580 con 0x560a5597a9c0 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.058+0000 7fc75effd640 1 -- 172.21.15.26:0/1081397083 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc778003d10 con 0x560a5597a9c0 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.058+0000 7fc75effd640 1 -- 172.21.15.26:0/1081397083 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc7700bde30 con 0x560a5597a9c0 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.058+0000 7fc788678740 10 client.15600 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.058+0000 7fc788678740 10 client.15600 resolve_mds: validated gid 24496 aka daemon mds.i 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.058+0000 7fc788678740 20 client.15600 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.058+0000 7fc788678740 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x560a55993ad0 0x560a55995f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.058+0000 7fc788678740 4 client.15600 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.058+0000 7fc788678740 1 -- 172.21.15.26:0/1081397083 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a558ecb10 con 0x560a55993ad0 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.059+0000 7fc77effd640 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x560a55993ad0 0x560a55995f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.059+0000 7fc75effd640 10 client.15600 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.059+0000 7fc77effd640 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x560a55993ad0 0x560a55995f20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e4f2df6e7696133 server_cookie=dc3cda25cd3bf362 in_seq=0 out_seq=0 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc75effd640 1 -- 172.21.15.26:0/1081397083 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a558ecb10 con 0x560a55993ad0 2024-04-21T10:31:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc75effd640 10 client.15600 handle_command_reply: tid=0 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc788678740 1 client.15600 shutdown 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc788678740 1 -- 172.21.15.26:0/1081397083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 msgr2=0x560a55980eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc788678740 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a55980eb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc7700f02d0 tx=0x7fc7700be5a0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc788678740 1 -- 172.21.15.26:0/1081397083 shutdown_connections 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc788678740 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x560a55993ad0 0x560a55995f20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc788678740 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a55983ec0 0x560a55981930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc75effd640 0 client.15600 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc788678740 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a559813f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc788678740 1 --2- 172.21.15.26:0/1081397083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a55980eb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc788678740 1 -- 172.21.15.26:0/1081397083 >> 172.21.15.26:0/1081397083 conn(0x560a559750f0 msgr2=0x560a55976e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc788678740 1 -- 172.21.15.26:0/1081397083 shutdown_connections 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc788678740 1 -- 172.21.15.26:0/1081397083 wait complete. 2024-04-21T10:31:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.060+0000 7fc788678740 20 client.15600 trim_cache size 0 max 16384 2024-04-21T10:31:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc788678740 1 Processor -- start 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc788678740 1 -- start start 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc788678740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a559835e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc788678740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a55973fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc788678740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a55983ec0 0x560a5597f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc788678740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801afed0 con 0x560a5597a9c0 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc788678740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801add20 con 0x560a55983ec0 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc788678740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801af800 con 0x560a5597b2e0 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc77f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a559835e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc77effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a55973fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc77f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a559835e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41196/0 (socket says 172.21.15.26:41196) 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc77f7fe640 1 -- 172.21.15.26:0/555950968 learned_addr learned my addr 172.21.15.26:0/555950968 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.086+0000 7fc77ffff640 1 --2- 172.21.15.26:0/555950968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a55983ec0 0x560a5597f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.087+0000 7fc77f7fe640 1 -- 172.21.15.26:0/555950968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 msgr2=0x560a55973fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.087+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/555950968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a55973fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.087+0000 7fc77f7fe640 1 -- 172.21.15.26:0/555950968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a55983ec0 msgr2=0x560a5597f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:23.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.087+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/555950968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a55983ec0 0x560a5597f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.087+0000 7fc77f7fe640 1 -- 172.21.15.26:0/555950968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a559709b0 con 0x560a5597a9c0 2024-04-21T10:31:23.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.087+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/555950968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a559835e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc7700bae40 tx=0x7fc7700035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b72f1c200f1eef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:23.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.087+0000 7fc7657fa640 1 -- 172.21.15.26:0/555950968 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700be7b0 con 0x560a5597a9c0 2024-04-21T10:31:23.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.087+0000 7fc7657fa640 1 -- 172.21.15.26:0/555950968 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7700be9e0 con 0x560a5597a9c0 2024-04-21T10:31:23.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.087+0000 7fc7657fa640 1 -- 172.21.15.26:0/555950968 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770089510 con 0x560a5597a9c0 2024-04-21T10:31:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.088+0000 7fc788678740 1 -- 172.21.15.26:0/555950968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 msgr2=0x560a559835e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.088+0000 7fc788678740 1 --2- 172.21.15.26:0/555950968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a559835e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc7700bae40 tx=0x7fc7700035f0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.088+0000 7fc788678740 1 -- 172.21.15.26:0/555950968 shutdown_connections 2024-04-21T10:31:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.088+0000 7fc788678740 1 --2- 172.21.15.26:0/555950968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a55983ec0 0x560a5597f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.088+0000 7fc788678740 1 --2- 172.21.15.26:0/555950968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a5597b2e0 0x560a55973fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.088+0000 7fc788678740 1 --2- 172.21.15.26:0/555950968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a559835e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.088+0000 7fc788678740 1 -- 172.21.15.26:0/555950968 >> 172.21.15.26:0/555950968 conn(0x560a559750f0 msgr2=0x560a55976bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.088+0000 7fc788678740 1 -- 172.21.15.26:0/555950968 shutdown_connections 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.088+0000 7fc788678740 1 -- 172.21.15.26:0/555950968 wait complete. 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.088+0000 7fc788678740 1 -- start start 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc788678740 1 Processor -- start 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc788678740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a55987f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc788678740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597b2e0 0x560a559884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc788678740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a55983ec0 0x560a559f9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc788678740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801aea60 con 0x560a5597a9c0 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc788678740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801ae390 con 0x560a5597b2e0 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc788678740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801af130 con 0x560a55983ec0 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc788678740 10 client.0 _ll_register_callbacks cb 0x7fc7700bad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc77f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a55987f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc77f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a55987f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41210/0 (socket says 172.21.15.26:41210) 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc77f7fe640 1 -- 172.21.15.26:0/1073382568 learned_addr learned my addr 172.21.15.26:0/1073382568 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc77effd640 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597b2e0 0x560a559884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc77ffff640 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a55983ec0 0x560a559f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc77f7fe640 1 -- 172.21.15.26:0/1073382568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a55983ec0 msgr2=0x560a559f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a55983ec0 0x560a559f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc77f7fe640 1 -- 172.21.15.26:0/1073382568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597b2e0 msgr2=0x560a559884c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597b2e0 0x560a559884c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc77f7fe640 1 -- 172.21.15.26:0/1073382568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a559f95a0 con 0x560a5597a9c0 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc77effd640 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597b2e0 0x560a559884c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.089+0000 7fc77ffff640 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a55983ec0 0x560a559f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc77f7fe640 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a55987f80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc7700be6e0 tx=0x7fc770005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2220692e73319799 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc75effd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc75effd640 1 -- 172.21.15.26:0/1073382568 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700be7b0 con 0x560a5597a9c0 2024-04-21T10:31:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc75effd640 1 -- 172.21.15.26:0/1073382568 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fc7700894c0 con 0x560a5597a9c0 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc788678740 1 -- 172.21.15.26:0/1073382568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7700be9e0 con 0x560a5597a9c0 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc75effd640 1 -- 172.21.15.26:0/1073382568 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc770005560 con 0x560a5597a9c0 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc788678740 10 client.15606 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc788678740 1 -- 172.21.15.26:0/1073382568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560a559f95a0 con 0x560a5597a9c0 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc75effd640 1 -- 172.21.15.26:0/1073382568 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7700be9e0 con 0x560a5597a9c0 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc75effd640 1 -- 172.21.15.26:0/1073382568 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc7700bd8a0 con 0x560a5597a9c0 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc788678740 10 client.15606 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc788678740 10 client.15606 resolve_mds: validated gid 24496 aka daemon mds.i 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc788678740 20 client.15606 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc788678740 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x560a559faba0 0x560a559fcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc788678740 4 client.15606 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 78.95, "await": true}] 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.090+0000 7fc788678740 1 -- 172.21.15.26:0/1073382568 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 78.95, "await": true}) -- 0x560a558e6980 con 0x560a559faba0 2024-04-21T10:31:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.091+0000 7fc77effd640 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x560a559faba0 0x560a559fcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.091+0000 7fc77effd640 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x560a559faba0 0x560a559fcf90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47398b4b0923e25 server_cookie=d507e7b09fdcfdd in_seq=0 out_seq=0 2024-04-21T10:31:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.091+0000 7fc75effd640 10 client.15606 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.104+0000 7fc75effd640 1 -- 172.21.15.26:0/1073382568 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x560a558e6980 con 0x560a559faba0 2024-04-21T10:31:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.104+0000 7fc75effd640 10 client.15606 handle_command_reply: tid=0 2024-04-21T10:31:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc788678740 1 client.15606 shutdown 2024-04-21T10:31:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc788678740 1 -- 172.21.15.26:0/1073382568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 msgr2=0x560a55987f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc788678740 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a55987f80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc7700be6e0 tx=0x7fc770005bd0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc788678740 1 -- 172.21.15.26:0/1073382568 shutdown_connections 2024-04-21T10:31:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc788678740 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x560a559faba0 0x560a559fcf90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc788678740 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560a55983ec0 0x560a559f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc75effd640 0 client.15606 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc788678740 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560a5597b2e0 0x560a559884c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc788678740 1 --2- 172.21.15.26:0/1073382568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560a5597a9c0 0x560a55987f80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc788678740 1 -- 172.21.15.26:0/1073382568 >> 172.21.15.26:0/1073382568 conn(0x560a559750f0 msgr2=0x560a55982560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc788678740 1 -- 172.21.15.26:0/1073382568 shutdown_connections 2024-04-21T10:31:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc788678740 1 -- 172.21.15.26:0/1073382568 wait complete. 2024-04-21T10:31:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.105+0000 7fc788678740 20 client.15606 trim_cache size 0 max 16384 2024-04-21T10:31:23.106 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:31:23.106 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 21, 2024-04-21T10:31:23.106 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24496, 2024-04-21T10:31:23.106 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 2, 2024-04-21T10:31:23.106 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-04-21T10:31:23.106 INFO:teuthology.orchestra.run.smithi026.stdout: "9e02290a": { 2024-04-21T10:31:23.106 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 2, 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 30.0, 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 79.0, 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:23.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:23.108 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:23.108 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:23.108 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:31:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.106+0000 7fc785a81640 1 -- 172.21.15.26:0/1268793269 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc7680a3f00 msgr2=0x7fc7680a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.106+0000 7fc785a81640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc7680a3f00 0x7fc7680a63c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc7801045e0 tx=0x7fc77400b390 comp rx=0 tx=0).stop 2024-04-21T10:31:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.106+0000 7fc785a81640 1 -- 172.21.15.26:0/1268793269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101110 msgr2=0x7fc780102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.106+0000 7fc785a81640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101110 0x7fc780102b50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc770005940 tx=0x7fc7700045e0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.106+0000 7fc785a81640 1 -- 172.21.15.26:0/1268793269 shutdown_connections 2024-04-21T10:31:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.106+0000 7fc785a81640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc7680a3f00 0x7fc7680a63c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.106+0000 7fc785a81640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7801035d0 0x7fc7801a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.106+0000 7fc785a81640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc780106190 0x7fc780103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.106+0000 7fc785a81640 1 --2- 172.21.15.26:0/1268793269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101110 0x7fc780102b50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.106+0000 7fc785a81640 1 -- 172.21.15.26:0/1268793269 >> 172.21.15.26:0/1268793269 conn(0x7fc780077fb0 msgr2=0x7fc78006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.106+0000 7fc785a81640 1 -- 172.21.15.26:0/1268793269 shutdown_connections 2024-04-21T10:31:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:23.106+0000 7fc785a81640 1 -- 172.21.15.26:0/1268793269 wait complete. 2024-04-21T10:31:23.117 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: 9e02290a, quiesce duration: 18.95 2024-04-21T10:31:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:23 smithi026 ceph-mon[27965]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-21T10:31:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:23 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/4107709893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:23.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:23 smithi169 ceph-mon[31469]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-21T10:31:23.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:23 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/4107709893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:23.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:23 smithi190 ceph-mon[31423]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-21T10:31:23.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:23 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/4107709893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:25.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:25 smithi026 ceph-mon[27965]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-21T10:31:25.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:25 smithi169 ceph-mon[31469]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-21T10:31:25.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:25 smithi190 ceph-mon[31423]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-21T10:31:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.941+0000 7f93d2c77640 1 -- 172.21.15.26:0/3634339583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93cc1010f0 msgr2=0x7f93cc1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.941+0000 7f93d2c77640 1 --2- 172.21.15.26:0/3634339583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93cc1010f0 0x7f93cc1014d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009860 tx=0x7f93c002bb70 comp rx=0 tx=0).stop 2024-04-21T10:31:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.941+0000 7f93d2c77640 1 -- 172.21.15.26:0/3634339583 shutdown_connections 2024-04-21T10:31:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.941+0000 7f93d2c77640 1 --2- 172.21.15.26:0/3634339583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93cc106170 0x7f93cc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.941+0000 7f93d2c77640 1 --2- 172.21.15.26:0/3634339583 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93cc101aa0 0x7f93cc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.941+0000 7f93d2c77640 1 --2- 172.21.15.26:0/3634339583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93cc1010f0 0x7f93cc1014d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.941+0000 7f93d2c77640 1 -- 172.21.15.26:0/3634339583 >> 172.21.15.26:0/3634339583 conn(0x7f93cc077fd0 msgr2=0x7f93cc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.941+0000 7f93d2c77640 1 -- 172.21.15.26:0/3634339583 shutdown_connections 2024-04-21T10:31:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.941+0000 7f93d2c77640 1 -- 172.21.15.26:0/3634339583 wait complete. 2024-04-21T10:31:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.943+0000 7f93d2c77640 1 Processor -- start 2024-04-21T10:31:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.943+0000 7f93d2c77640 1 -- start start 2024-04-21T10:31:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.943+0000 7f93d2c77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93cc1010f0 0x7f93cc197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.943+0000 7f93d2c77640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93cc106170 0x7f93cc197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.943+0000 7f93d2c77640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93cc19bee0 0x7f93cc19c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.943+0000 7f93d0a13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93cc1010f0 0x7f93cc197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.943+0000 7f93d2c77640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc111080 con 0x7f93cc1010f0 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.943+0000 7f93d2c77640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc110f00 con 0x7f93cc106170 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.943+0000 7f93d2c77640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc111200 con 0x7f93cc19bee0 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.943+0000 7f93d0a13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93cc1010f0 0x7f93cc197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39974/0 (socket says 172.21.15.26:39974) 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.943+0000 7f93d0a13640 1 -- 172.21.15.26:0/2244648214 learned_addr learned my addr 172.21.15.26:0/2244648214 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.943+0000 7f93cbfff640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93cc106170 0x7f93cc197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.944+0000 7f93d1214640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93cc19bee0 0x7f93cc19c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.944+0000 7f93cbfff640 1 -- 172.21.15.26:0/2244648214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93cc19bee0 msgr2=0x7f93cc19c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.944+0000 7f93cbfff640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93cc19bee0 0x7f93cc19c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.944+0000 7f93cbfff640 1 -- 172.21.15.26:0/2244648214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93cc1010f0 msgr2=0x7f93cc197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.944+0000 7f93cbfff640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93cc1010f0 0x7f93cc197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.944+0000 7f93cbfff640 1 -- 172.21.15.26:0/2244648214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93cc19c9d0 con 0x7f93cc106170 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.944+0000 7f93d0a13640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93cc1010f0 0x7f93cc197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.944+0000 7f93cbfff640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93cc106170 0x7f93cc197b60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009b20 tx=0x7f93b800b2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.944+0000 7f93c9ffb640 1 -- 172.21.15.26:0/2244648214 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b801e070 con 0x7f93cc106170 2024-04-21T10:31:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.944+0000 7f93c9ffb640 1 -- 172.21.15.26:0/2244648214 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b800bd00 con 0x7f93cc106170 2024-04-21T10:31:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.944+0000 7f93d2c77640 1 -- 172.21.15.26:0/2244648214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93cc19ccc0 con 0x7f93cc106170 2024-04-21T10:31:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.945+0000 7f93d2c77640 1 -- 172.21.15.26:0/2244648214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93cc0698c0 con 0x7f93cc106170 2024-04-21T10:31:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.945+0000 7f93c9ffb640 1 -- 172.21.15.26:0/2244648214 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8019450 con 0x7f93cc106170 2024-04-21T10:31:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.948+0000 7f93c9ffb640 1 -- 172.21.15.26:0/2244648214 <== mon.2 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f93b8021070 con 0x7f93cc106170 2024-04-21T10:31:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.949+0000 7f93c9ffb640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f93bc0a3e50 0x7f93bc0a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.949+0000 7f93d0a13640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f93bc0a3e50 0x7f93bc0a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.949+0000 7f93d5862740 1 Processor -- start 2024-04-21T10:31:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.949+0000 7f93c9ffb640 1 -- 172.21.15.26:0/2244648214 <== mon.2 v2:172.21.15.169:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f93b80d42f0 con 0x7f93cc106170 2024-04-21T10:31:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.949+0000 7f93d5862740 1 -- start start 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.949+0000 7f93d5862740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 0x55dcde5f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.949+0000 7f93d5862740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 0x55dcde5f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93d5862740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93d5862740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dcde5508d0 con 0x55dcde5f4370 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93d5862740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dcde4ede80 con 0x55dcde5f50e0 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93d5862740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dcde54c890 con 0x55dcde5fd8f0 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 0x55dcde5f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 0x55dcde5f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39976/0 (socket says 172.21.15.26:39976) 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93d0a13640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f93bc0a3e50 0x7f93bc0a6310 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f93c0005e00 tx=0x7f93c00058b0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93cbfff640 1 -- 172.21.15.26:0/367249240 learned_addr learned my addr 172.21.15.26:0/367249240 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93d1214640 1 --2- 172.21.15.26:0/367249240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93cbfff640 1 -- 172.21.15.26:0/367249240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 msgr2=0x55dcde5ffce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93cbfff640 1 --2- 172.21.15.26:0/367249240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5ffce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93d0a13640 1 --2- 172.21.15.26:0/367249240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 0x55dcde5f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93cbfff640 1 -- 172.21.15.26:0/367249240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 msgr2=0x55dcde5f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93cbfff640 1 --2- 172.21.15.26:0/367249240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 0x55dcde5f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93cbfff640 1 -- 172.21.15.26:0/367249240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcde50f320 con 0x55dcde5f4370 2024-04-21T10:31:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93d0a13640 1 --2- 172.21.15.26:0/367249240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 0x55dcde5f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.950+0000 7f93cbfff640 1 --2- 172.21.15.26:0/367249240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 0x55dcde5f47f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f93b8010900 tx=0x7f93b809fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6af99368a3f62c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.951+0000 7f93b6ffd640 1 -- 172.21.15.26:0/367249240 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b809e410 con 0x55dcde5f4370 2024-04-21T10:31:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.951+0000 7f93b6ffd640 1 -- 172.21.15.26:0/367249240 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b80a40d0 con 0x55dcde5f4370 2024-04-21T10:31:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.951+0000 7f93b6ffd640 1 -- 172.21.15.26:0/367249240 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b809be60 con 0x55dcde5f4370 2024-04-21T10:31:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.952+0000 7f93d5862740 1 -- 172.21.15.26:0/367249240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 msgr2=0x55dcde5f47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.952+0000 7f93d5862740 1 --2- 172.21.15.26:0/367249240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 0x55dcde5f47f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f93b8010900 tx=0x7f93b809fb60 comp rx=0 tx=0).stop 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.952+0000 7f93d5862740 1 -- 172.21.15.26:0/367249240 shutdown_connections 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.952+0000 7f93d5862740 1 --2- 172.21.15.26:0/367249240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.952+0000 7f93d5862740 1 --2- 172.21.15.26:0/367249240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 0x55dcde5f47f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.952+0000 7f93d5862740 1 --2- 172.21.15.26:0/367249240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 0x55dcde5f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.952+0000 7f93d5862740 1 -- 172.21.15.26:0/367249240 >> 172.21.15.26:0/367249240 conn(0x55dcde5b2c90 msgr2=0x55dcde5ef060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.952+0000 7f93d5862740 1 -- 172.21.15.26:0/367249240 shutdown_connections 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.952+0000 7f93d5862740 1 -- 172.21.15.26:0/367249240 wait complete. 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.952+0000 7f93d5862740 1 -- start start 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.952+0000 7f93d5862740 1 Processor -- start 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.953+0000 7f93d5862740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f4370 0x55dcde5fd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.953+0000 7f93d5862740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 0x55dcde5fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.953+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 0x55dcde5fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.953+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 0x55dcde5fa6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39978/0 (socket says 172.21.15.26:39978) 2024-04-21T10:31:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.953+0000 7f93cbfff640 1 -- 172.21.15.26:0/1107313837 learned_addr learned my addr 172.21.15.26:0/1107313837 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.953+0000 7f93d5862740 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5fae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.953+0000 7f93d5862740 1 -- 172.21.15.26:0/1107313837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dcde600620 con 0x55dcde5f50e0 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.953+0000 7f93d5862740 1 -- 172.21.15.26:0/1107313837 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dcde4d8eb0 con 0x55dcde5f4370 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.953+0000 7f93d5862740 1 -- 172.21.15.26:0/1107313837 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55dcde6007a0 con 0x55dcde5fd8f0 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.953+0000 7f93d0a13640 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f4370 0x55dcde5fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.953+0000 7f93d1214640 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93cbfff640 1 -- 172.21.15.26:0/1107313837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 msgr2=0x55dcde5fae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93cbfff640 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5fae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93cbfff640 1 -- 172.21.15.26:0/1107313837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f4370 msgr2=0x55dcde5fd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93cbfff640 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f4370 0x55dcde5fd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93cbfff640 1 -- 172.21.15.26:0/1107313837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcde263210 con 0x55dcde5f50e0 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93d0a13640 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f4370 0x55dcde5fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93cbfff640 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 0x55dcde5fa6f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f93b800bfd0 tx=0x7f93b80d1830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c8e56a477b4759 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93abfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93d5862740 10 client.0 _ll_register_callbacks cb 0x55dcde505b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93d5862740 1 -- 172.21.15.26:0/1107313837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b80a40d0 con 0x55dcde5f50e0 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93abfff640 1 -- 172.21.15.26:0/1107313837 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b80bd070 con 0x55dcde5f50e0 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93abfff640 1 -- 172.21.15.26:0/1107313837 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f93b809f760 con 0x55dcde5f50e0 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93abfff640 1 -- 172.21.15.26:0/1107313837 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f93b809e4d0 con 0x55dcde5f50e0 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93d5862740 10 client.15618 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.954+0000 7f93d5862740 1 -- 172.21.15.26:0/1107313837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55dcde32f620 con 0x55dcde5f50e0 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.955+0000 7f93abfff640 1 -- 172.21.15.26:0/1107313837 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f93b80b9260 con 0x55dcde5f50e0 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.955+0000 7f93d5862740 10 client.15618 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.955+0000 7f93d5862740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.955+0000 7f93d5862740 20 client.15618 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.955+0000 7f93d5862740 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55dcde60d280 0x55dcde60f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.955+0000 7f93d5862740 4 client.15618 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:31:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.955+0000 7f93d5862740 1 -- 172.21.15.26:0/1107313837 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcde564880 con 0x55dcde60d280 2024-04-21T10:31:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.955+0000 7f93d0a13640 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55dcde60d280 0x55dcde60f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.956+0000 7f93d0a13640 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55dcde60d280 0x55dcde60f670 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab4784af1ba79e8 server_cookie=9805f24afcb6d229 in_seq=0 out_seq=0 2024-04-21T10:31:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.956+0000 7f93abfff640 10 client.15618 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.956+0000 7f93abfff640 1 -- 172.21.15.26:0/1107313837 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dcde564880 con 0x55dcde60d280 2024-04-21T10:31:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.956+0000 7f93abfff640 10 client.15618 handle_command_reply: tid=0 2024-04-21T10:31:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.956+0000 7f93d5862740 1 client.15618 shutdown 2024-04-21T10:31:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.957+0000 7f93d5862740 1 -- 172.21.15.26:0/1107313837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 msgr2=0x55dcde5fa6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.957+0000 7f93d5862740 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 0x55dcde5fa6f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f93b800bfd0 tx=0x7f93b80d1830 comp rx=0 tx=0).stop 2024-04-21T10:31:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.957+0000 7f93d5862740 1 -- 172.21.15.26:0/1107313837 shutdown_connections 2024-04-21T10:31:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.957+0000 7f93d5862740 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55dcde60d280 0x55dcde60f670 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.957+0000 7f93d5862740 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5fae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.957+0000 7f93d5862740 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 0x55dcde5fa6f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.957+0000 7f93d5862740 1 --2- 172.21.15.26:0/1107313837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f4370 0x55dcde5fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.957+0000 7f93d5862740 1 -- 172.21.15.26:0/1107313837 >> 172.21.15.26:0/1107313837 conn(0x55dcde5b2c90 msgr2=0x55dcde5f0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.957+0000 7f93abfff640 0 client.15618 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.957+0000 7f93d5862740 1 -- 172.21.15.26:0/1107313837 shutdown_connections 2024-04-21T10:31:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.957+0000 7f93d5862740 1 -- 172.21.15.26:0/1107313837 wait complete. 2024-04-21T10:31:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.957+0000 7f93d5862740 20 client.15618 trim_cache size 0 max 16384 2024-04-21T10:31:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.987+0000 7f93d5862740 1 Processor -- start 2024-04-21T10:31:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.987+0000 7f93d5862740 1 -- start start 2024-04-21T10:31:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.987+0000 7f93d5862740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 0x55dcde5fa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.987+0000 7f93d5862740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 0x55dcde609520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.987+0000 7f93d5862740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.987+0000 7f93d5862740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1ade90 con 0x55dcde5f4370 2024-04-21T10:31:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.987+0000 7f93d5862740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1ab760 con 0x55dcde5f50e0 2024-04-21T10:31:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.987+0000 7f93d5862740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1a95b0 con 0x55dcde5fd8f0 2024-04-21T10:31:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93d0a13640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 0x55dcde5fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 0x55dcde609520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 0x55dcde609520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.26:53822/0 (socket says 172.21.15.26:53822) 2024-04-21T10:31:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93cbfff640 1 -- 172.21.15.26:0/2626206860 learned_addr learned my addr 172.21.15.26:0/2626206860 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93cbfff640 1 -- 172.21.15.26:0/2626206860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 msgr2=0x55dcde5fce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93d1214640 1 --2- 172.21.15.26:0/2626206860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93cbfff640 1 --2- 172.21.15.26:0/2626206860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5fce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93cbfff640 1 -- 172.21.15.26:0/2626206860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 msgr2=0x55dcde5fa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93cbfff640 1 --2- 172.21.15.26:0/2626206860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 0x55dcde5fa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93cbfff640 1 -- 172.21.15.26:0/2626206860 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcde5f3640 con 0x55dcde5f50e0 2024-04-21T10:31:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93d0a13640 1 --2- 172.21.15.26:0/2626206860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 0x55dcde5fa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93d1214640 1 --2- 172.21.15.26:0/2626206860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.988+0000 7f93cbfff640 1 --2- 172.21.15.26:0/2626206860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 0x55dcde609520 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f93b809cae0 tx=0x7f93b80b7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34292deba6678160 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.989+0000 7f93b67fc640 1 -- 172.21.15.26:0/2626206860 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b809f560 con 0x55dcde5f50e0 2024-04-21T10:31:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.989+0000 7f93b67fc640 1 -- 172.21.15.26:0/2626206860 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b80a40d0 con 0x55dcde5f50e0 2024-04-21T10:31:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.989+0000 7f93b67fc640 1 -- 172.21.15.26:0/2626206860 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b809be60 con 0x55dcde5f50e0 2024-04-21T10:31:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.989+0000 7f93d5862740 1 -- 172.21.15.26:0/2626206860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 msgr2=0x55dcde609520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.990+0000 7f93d5862740 1 --2- 172.21.15.26:0/2626206860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 0x55dcde609520 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f93b809cae0 tx=0x7f93b80b7000 comp rx=0 tx=0).stop 2024-04-21T10:31:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.990+0000 7f93d5862740 1 -- 172.21.15.26:0/2626206860 shutdown_connections 2024-04-21T10:31:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.990+0000 7f93d5862740 1 --2- 172.21.15.26:0/2626206860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.990+0000 7f93d5862740 1 --2- 172.21.15.26:0/2626206860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f50e0 0x55dcde609520 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.990+0000 7f93d5862740 1 --2- 172.21.15.26:0/2626206860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f4370 0x55dcde5fa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.990+0000 7f93d5862740 1 -- 172.21.15.26:0/2626206860 >> 172.21.15.26:0/2626206860 conn(0x55dcde5b2c90 msgr2=0x55dcde5f9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.990+0000 7f93d5862740 1 -- 172.21.15.26:0/2626206860 shutdown_connections 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.990+0000 7f93d5862740 1 -- 172.21.15.26:0/2626206860 wait complete. 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.990+0000 7f93d5862740 1 -- start start 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.990+0000 7f93d5862740 1 Processor -- start 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93d5862740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f4370 0x55dcde605a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93d5862740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 0x55dcde605f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93d5862740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5f7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93d5862740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1a9c20 con 0x55dcde5f50e0 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93d5862740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1ab090 con 0x55dcde5f4370 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93d5862740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1aa2f0 con 0x55dcde5fd8f0 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93d5862740 10 client.0 _ll_register_callbacks cb 0x7f93b809ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 0x55dcde605f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93cbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 0x55dcde605f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40002/0 (socket says 172.21.15.26:40002) 2024-04-21T10:31:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93cbfff640 1 -- 172.21.15.26:0/96751225 learned_addr learned my addr 172.21.15.26:0/96751225 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93d0a13640 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f4370 0x55dcde605a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93d1214640 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93cbfff640 1 -- 172.21.15.26:0/96751225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 msgr2=0x55dcde5f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93cbfff640 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93cbfff640 1 -- 172.21.15.26:0/96751225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f4370 msgr2=0x55dcde605a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93cbfff640 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f4370 0x55dcde605a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93cbfff640 1 -- 172.21.15.26:0/96751225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcde5f7eb0 con 0x55dcde5f50e0 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.991+0000 7f93d1214640 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.992+0000 7f93cbfff640 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 0x55dcde605f70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f93b8003950 tx=0x7f93b809f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3e12e0e19cf57d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.992+0000 7f93abfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.992+0000 7f93abfff640 1 -- 172.21.15.26:0/96751225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b80bd020 con 0x55dcde5f50e0 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.992+0000 7f93abfff640 1 -- 172.21.15.26:0/96751225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f93b80892b0 con 0x55dcde5f50e0 2024-04-21T10:31:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.992+0000 7f93d5862740 1 -- 172.21.15.26:0/96751225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b80a40d0 con 0x55dcde5f50e0 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.992+0000 7f93abfff640 1 -- 172.21.15.26:0/96751225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f93b80a4410 con 0x55dcde5f50e0 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.992+0000 7f93d5862740 10 client.15624 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.992+0000 7f93d5862740 1 -- 172.21.15.26:0/96751225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dcde5f7eb0 con 0x55dcde5f50e0 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.992+0000 7f93abfff640 1 -- 172.21.15.26:0/96751225 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b809efd0 con 0x55dcde5f50e0 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.993+0000 7f93abfff640 1 -- 172.21.15.26:0/96751225 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f93b80d1d30 con 0x55dcde5f50e0 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.993+0000 7f93d5862740 10 client.15624 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.993+0000 7f93d5862740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.993+0000 7f93d5862740 20 client.15624 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.993+0000 7f93d5862740 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55dcde5fe670 0x55dcde6757e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.993+0000 7f93d5862740 4 client.15624 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.993+0000 7f93d5862740 1 -- 172.21.15.26:0/96751225 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f93b80a40d0 con 0x55dcde5fe670 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.993+0000 7f93d0a13640 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55dcde5fe670 0x55dcde6757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.994+0000 7f93d0a13640 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55dcde5fe670 0x55dcde6757e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b127ce27940290d3 server_cookie=34d33e0eddd256d2 in_seq=0 out_seq=0 2024-04-21T10:31:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:26.994+0000 7f93abfff640 10 client.15624 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.001+0000 7f93abfff640 1 -- 172.21.15.26:0/96751225 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f93b80a40d0 con 0x55dcde5fe670 2024-04-21T10:31:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.001+0000 7f93abfff640 10 client.15624 handle_command_reply: tid=0 2024-04-21T10:31:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.001+0000 7f93d5862740 1 client.15624 shutdown 2024-04-21T10:31:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.001+0000 7f93d5862740 1 -- 172.21.15.26:0/96751225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 msgr2=0x55dcde605f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.001+0000 7f93d5862740 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 0x55dcde605f70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f93b8003950 tx=0x7f93b809f810 comp rx=0 tx=0).stop 2024-04-21T10:31:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.002+0000 7f93d5862740 1 -- 172.21.15.26:0/96751225 shutdown_connections 2024-04-21T10:31:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.002+0000 7f93d5862740 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55dcde5fe670 0x55dcde6757e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.002+0000 7f93d5862740 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dcde5fd8f0 0x55dcde5f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.002+0000 7f93d5862740 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcde5f50e0 0x55dcde605f70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.002+0000 7f93d5862740 1 --2- 172.21.15.26:0/96751225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcde5f4370 0x55dcde605a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.002+0000 7f93d5862740 1 -- 172.21.15.26:0/96751225 >> 172.21.15.26:0/96751225 conn(0x55dcde5b2c90 msgr2=0x55dcde5e58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.002+0000 7f93abfff640 0 client.15624 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.002+0000 7f93d5862740 1 -- 172.21.15.26:0/96751225 shutdown_connections 2024-04-21T10:31:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.002+0000 7f93d5862740 1 -- 172.21.15.26:0/96751225 wait complete. 2024-04-21T10:31:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.002+0000 7f93d5862740 20 client.15624 trim_cache size 0 max 16384 2024-04-21T10:31:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-04-21T10:31:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-04-21T10:31:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-04-21T10:31:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-04-21T10:31:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.003+0000 7f93d2c77640 1 -- 172.21.15.26:0/2244648214 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f93bc0a3e50 msgr2=0x7f93bc0a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.003+0000 7f93d2c77640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f93bc0a3e50 0x7f93bc0a6310 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f93c0005e00 tx=0x7f93c00058b0 comp rx=0 tx=0).stop 2024-04-21T10:31:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.003+0000 7f93d2c77640 1 -- 172.21.15.26:0/2244648214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93cc106170 msgr2=0x7f93cc197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.003+0000 7f93d2c77640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93cc106170 0x7f93cc197b60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009b20 tx=0x7f93b800b2c0 comp rx=0 tx=0).stop 2024-04-21T10:31:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.003+0000 7f93d2c77640 1 -- 172.21.15.26:0/2244648214 shutdown_connections 2024-04-21T10:31:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.003+0000 7f93d2c77640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f93bc0a3e50 0x7f93bc0a6310 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.003+0000 7f93d2c77640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93cc19bee0 0x7f93cc19c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.003+0000 7f93d2c77640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93cc106170 0x7f93cc197b60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.003+0000 7f93d2c77640 1 --2- 172.21.15.26:0/2244648214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93cc1010f0 0x7f93cc197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.003+0000 7f93d2c77640 1 -- 172.21.15.26:0/2244648214 >> 172.21.15.26:0/2244648214 conn(0x7f93cc077fd0 msgr2=0x7f93cc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.003+0000 7f93d2c77640 1 -- 172.21.15.26:0/2244648214 shutdown_connections 2024-04-21T10:31:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:27.003+0000 7f93d2c77640 1 -- 172.21.15.26:0/2244648214 wait complete. 2024-04-21T10:31:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:27 smithi026 ceph-mon[27965]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-21T10:31:27.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:27 smithi169 ceph-mon[31469]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-21T10:31:27.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:27 smithi190 ceph-mon[31423]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-21T10:31:29.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:29 smithi026 ceph-mon[27965]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-21T10:31:29.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:29 smithi169 ceph-mon[31469]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-21T10:31:29.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:29 smithi190 ceph-mon[31423]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-21T10:31:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:31 smithi026 ceph-mon[27965]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-21T10:31:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:31 smithi169 ceph-mon[31469]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-21T10:31:31.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:31 smithi190 ceph-mon[31423]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-21T10:31:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:33 smithi026 ceph-mon[27965]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-21T10:31:33.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:33 smithi169 ceph-mon[31469]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-21T10:31:33.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:33 smithi190 ceph-mon[31423]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-21T10:31:35.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:35 smithi026 ceph-mon[27965]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-21T10:31:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:35 smithi169 ceph-mon[31469]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-21T10:31:35.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:35 smithi190 ceph-mon[31423]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-21T10:31:37.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:37 smithi026 ceph-mon[27965]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-21T10:31:37.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:37 smithi169 ceph-mon[31469]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-21T10:31:37.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:37 smithi190 ceph-mon[31423]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-21T10:31:39.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:39 smithi026 ceph-mon[27965]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-21T10:31:39.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:39 smithi169 ceph-mon[31469]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-21T10:31:39.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:39 smithi190 ceph-mon[31423]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-21T10:31:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.142+0000 7f040d281640 1 -- 172.21.15.26:0/577698823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0408106170 msgr2=0x7f040810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.142+0000 7f040d281640 1 --2- 172.21.15.26:0/577698823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0408106170 0x7f040810dd40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0400009ca0 tx=0x7f040002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.143+0000 7f040d281640 1 -- 172.21.15.26:0/577698823 shutdown_connections 2024-04-21T10:31:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.143+0000 7f040d281640 1 --2- 172.21.15.26:0/577698823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0408106170 0x7f040810dd40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.143+0000 7f040d281640 1 --2- 172.21.15.26:0/577698823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0408101aa0 0x7f0408105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.143+0000 7f040d281640 1 --2- 172.21.15.26:0/577698823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04081010f0 0x7f04081014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.143+0000 7f040d281640 1 -- 172.21.15.26:0/577698823 >> 172.21.15.26:0/577698823 conn(0x7f0408077fd0 msgr2=0x7f04080783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.143+0000 7f040d281640 1 -- 172.21.15.26:0/577698823 shutdown_connections 2024-04-21T10:31:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.143+0000 7f040d281640 1 -- 172.21.15.26:0/577698823 wait complete. 2024-04-21T10:31:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f040d281640 1 Processor -- start 2024-04-21T10:31:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f040d281640 1 -- start start 2024-04-21T10:31:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f040d281640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04081010f0 0x7f0408199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f040d281640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0408106170 0x7f04081999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f040d281640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0408199f50 0x7f04081a4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f040d281640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04081110c0 con 0x7f0408199f50 2024-04-21T10:31:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f040d281640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0408110f40 con 0x7f04081010f0 2024-04-21T10:31:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f040d281640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0408111240 con 0x7f0408106170 2024-04-21T10:31:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f04077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0408199f50 0x7f04081a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f04077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0408199f50 0x7f04081a4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57712/0 (socket says 172.21.15.26:57712) 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f04077fe640 1 -- 172.21.15.26:0/3235355618 learned_addr learned my addr 172.21.15.26:0/3235355618 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f03fffff640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0408106170 0x7f04081999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f0406ffd640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04081010f0 0x7f0408199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.144+0000 7f04077fe640 1 -- 172.21.15.26:0/3235355618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0408106170 msgr2=0x7f04081999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.145+0000 7f04077fe640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0408106170 0x7f04081999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.145+0000 7f04077fe640 1 -- 172.21.15.26:0/3235355618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04081010f0 msgr2=0x7f0408199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.145+0000 7f04077fe640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04081010f0 0x7f0408199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.145+0000 7f04077fe640 1 -- 172.21.15.26:0/3235355618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04081a4cd0 con 0x7f0408199f50 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.145+0000 7f03fffff640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0408106170 0x7f04081999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.145+0000 7f04077fe640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0408199f50 0x7f04081a4500 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0400005e30 tx=0x7f0400034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.145+0000 7f0404ff9640 1 -- 172.21.15.26:0/3235355618 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400041020 con 0x7f0408199f50 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.145+0000 7f0404ff9640 1 -- 172.21.15.26:0/3235355618 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0400032970 con 0x7f0408199f50 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.145+0000 7f040d281640 1 -- 172.21.15.26:0/3235355618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04081a4f60 con 0x7f0408199f50 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.145+0000 7f0404ff9640 1 -- 172.21.15.26:0/3235355618 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400039040 con 0x7f0408199f50 2024-04-21T10:31:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.145+0000 7f040d281640 1 -- 172.21.15.26:0/3235355618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04081a5470 con 0x7f0408199f50 2024-04-21T10:31:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.149+0000 7f040fe76740 1 Processor -- start 2024-04-21T10:31:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.149+0000 7f0404ff9640 1 -- 172.21.15.26:0/3235355618 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f0400044020 con 0x7f0408199f50 2024-04-21T10:31:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.149+0000 7f040fe76740 1 -- start start 2024-04-21T10:31:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.149+0000 7f040fe76740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592b8d0 0x561e6592a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.149+0000 7f040fe76740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e6592afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.149+0000 7f040fe76740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e659340e0 0x561e659364d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.149+0000 7f040fe76740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561e6588f8d0 con 0x561e6592b8d0 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.149+0000 7f040fe76740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561e6582ce80 con 0x561e6592ab60 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.149+0000 7f040fe76740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561e6588b890 con 0x561e659340e0 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f03fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592b8d0 0x561e6592a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f0406ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e6592afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f0406ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e6592afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.26:50604/0 (socket says 172.21.15.26:50604) 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f03fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592b8d0 0x561e6592a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57716/0 (socket says 172.21.15.26:57716) 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f0406ffd640 1 -- 172.21.15.26:0/2867031658 learned_addr learned my addr 172.21.15.26:0/2867031658 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f0406ffd640 1 -- 172.21.15.26:0/2867031658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e659340e0 msgr2=0x561e659364d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f0406ffd640 1 --2- 172.21.15.26:0/2867031658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e659340e0 0x561e659364d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f0406ffd640 1 -- 172.21.15.26:0/2867031658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592b8d0 msgr2=0x561e6592a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f0406ffd640 1 --2- 172.21.15.26:0/2867031658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592b8d0 0x561e6592a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f0406ffd640 1 -- 172.21.15.26:0/2867031658 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6584e320 con 0x561e6592ab60 2024-04-21T10:31:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f03fffff640 1 --2- 172.21.15.26:0/2867031658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592b8d0 0x561e6592a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f0404ff9640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f03f40a3ff0 0x7f03f40a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f0404ff9640 1 -- 172.21.15.26:0/3235355618 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f04000f2720 con 0x7f0408199f50 2024-04-21T10:31:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f0406ffd640 1 --2- 172.21.15.26:0/2867031658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e6592afe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f03f800bc90 tx=0x7f03f8012b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead1eb8bcc3a3bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.150+0000 7f03fffff640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f03f40a3ff0 0x7f03f40a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.151+0000 7f03fd7fa640 1 -- 172.21.15.26:0/2867031658 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8004030 con 0x561e6592ab60 2024-04-21T10:31:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.151+0000 7f03fd7fa640 1 -- 172.21.15.26:0/2867031658 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f800d920 con 0x561e6592ab60 2024-04-21T10:31:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.151+0000 7f03fd7fa640 1 -- 172.21.15.26:0/2867031658 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8003ad0 con 0x561e6592ab60 2024-04-21T10:31:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.152+0000 7f040fe76740 1 -- 172.21.15.26:0/2867031658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 msgr2=0x561e6592afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.152+0000 7f040fe76740 1 --2- 172.21.15.26:0/2867031658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e6592afe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f03f800bc90 tx=0x7f03f8012b30 comp rx=0 tx=0).stop 2024-04-21T10:31:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.152+0000 7f03fffff640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f03f40a3ff0 0x7f03f40a64b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x561e659269e0 tx=0x7f03f000b580 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.152+0000 7f040fe76740 1 -- 172.21.15.26:0/2867031658 shutdown_connections 2024-04-21T10:31:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.152+0000 7f040fe76740 1 --2- 172.21.15.26:0/2867031658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e659340e0 0x561e659364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.152+0000 7f040fe76740 1 --2- 172.21.15.26:0/2867031658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e6592afe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.152+0000 7f040fe76740 1 --2- 172.21.15.26:0/2867031658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592b8d0 0x561e6592a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.152+0000 7f040fe76740 1 -- 172.21.15.26:0/2867031658 >> 172.21.15.26:0/2867031658 conn(0x561e658f1c90 msgr2=0x561e65925850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.152+0000 7f040fe76740 1 -- 172.21.15.26:0/2867031658 shutdown_connections 2024-04-21T10:31:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.153+0000 7f040fe76740 1 -- 172.21.15.26:0/2867031658 wait complete. 2024-04-21T10:31:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.153+0000 7f040fe76740 1 -- start start 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.154+0000 7f040fe76740 1 Processor -- start 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.154+0000 7f040fe76740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e65933a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.154+0000 7f040fe76740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e6592b8d0 0x561e65930dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.154+0000 7f040fe76740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 0x561e65931320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.154+0000 7f040fe76740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561e65947dc0 con 0x561e659340e0 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.154+0000 7f040fe76740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561e65817eb0 con 0x561e6592ab60 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.154+0000 7f040fe76740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x561e65947f40 con 0x561e6592b8d0 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.154+0000 7f040fe76740 10 client.0 _ll_register_callbacks cb 0x561e65844b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.154+0000 7f04077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 0x561e65931320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.154+0000 7f04077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 0x561e65931320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57728/0 (socket says 172.21.15.26:57728) 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.154+0000 7f04077fe640 1 -- 172.21.15.26:0/804859209 learned_addr learned my addr 172.21.15.26:0/804859209 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.154+0000 7f0406ffd640 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e65933a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.155+0000 7f03fffff640 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e6592b8d0 0x561e65930dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.155+0000 7f04077fe640 1 -- 172.21.15.26:0/804859209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e6592b8d0 msgr2=0x561e65930dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.155+0000 7f04077fe640 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e6592b8d0 0x561e65930dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.155+0000 7f04077fe640 1 -- 172.21.15.26:0/804859209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 msgr2=0x561e65933a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.155+0000 7f04077fe640 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e65933a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.155+0000 7f04077fe640 1 -- 172.21.15.26:0/804859209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e655a2210 con 0x561e659340e0 2024-04-21T10:31:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.155+0000 7f04077fe640 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 0x561e65931320 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f04000be2a0 tx=0x7f04000be0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f89920503aacbe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.155+0000 7f03ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.155+0000 7f03ee7fc640 1 -- 172.21.15.26:0/804859209 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04000be5a0 con 0x561e659340e0 2024-04-21T10:31:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.155+0000 7f040fe76740 1 -- 172.21.15.26:0/804859209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f800d920 con 0x561e659340e0 2024-04-21T10:31:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.155+0000 7f03ee7fc640 1 -- 172.21.15.26:0/804859209 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f04000c6d80 con 0x561e659340e0 2024-04-21T10:31:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.155+0000 7f03ee7fc640 1 -- 172.21.15.26:0/804859209 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f04000c2950 con 0x561e659340e0 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.156+0000 7f040fe76740 10 client.15636 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.156+0000 7f040fe76740 1 -- 172.21.15.26:0/804859209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561e6566e620 con 0x561e659340e0 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.156+0000 7f03ee7fc640 1 -- 172.21.15.26:0/804859209 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f800d920 con 0x561e659340e0 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.156+0000 7f03ee7fc640 1 -- 172.21.15.26:0/804859209 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f040003c030 con 0x561e659340e0 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.156+0000 7f040fe76740 10 client.15636 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.156+0000 7f040fe76740 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.156+0000 7f040fe76740 20 client.15636 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.156+0000 7f040fe76740 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x561e65955050 0x561e659574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.156+0000 7f040fe76740 4 client.15636 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.156+0000 7f040fe76740 1 -- 172.21.15.26:0/804859209 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e658a3880 con 0x561e65955050 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.157+0000 7f0406ffd640 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x561e65955050 0x561e659574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.157+0000 7f0406ffd640 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x561e65955050 0x561e659574c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abae0b03aff4b6cd server_cookie=d1d7b1ab884785cf in_seq=0 out_seq=0 2024-04-21T10:31:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.157+0000 7f03ee7fc640 10 client.15636 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f03ee7fc640 1 -- 172.21.15.26:0/804859209 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e658a3880 con 0x561e65955050 2024-04-21T10:31:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f03ee7fc640 10 client.15636 handle_command_reply: tid=0 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f040fe76740 1 client.15636 shutdown 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f040fe76740 1 -- 172.21.15.26:0/804859209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 msgr2=0x561e65931320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f040fe76740 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 0x561e65931320 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f04000be2a0 tx=0x7f04000be0e0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f040fe76740 1 -- 172.21.15.26:0/804859209 shutdown_connections 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f040fe76740 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x561e65955050 0x561e659574c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f040fe76740 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 0x561e65931320 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f040fe76740 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e6592b8d0 0x561e65930dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f040fe76740 1 --2- 172.21.15.26:0/804859209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e65933a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f040fe76740 1 -- 172.21.15.26:0/804859209 >> 172.21.15.26:0/804859209 conn(0x561e658f1c90 msgr2=0x561e659275a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f03ee7fc640 0 client.15636 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f040fe76740 1 -- 172.21.15.26:0/804859209 shutdown_connections 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f040fe76740 1 -- 172.21.15.26:0/804859209 wait complete. 2024-04-21T10:31:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.158+0000 7f040fe76740 20 client.15636 trim_cache size 0 max 16384 2024-04-21T10:31:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.188+0000 7f040fe76740 1 Processor -- start 2024-04-21T10:31:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.188+0000 7f040fe76740 1 -- start start 2024-04-21T10:31:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f040fe76740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592ab60 0x561e65950cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f040fe76740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592b8d0 0x561e6592a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f040fe76740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e659340e0 0x561e659334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f040fe76740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04081a5920 con 0x561e6592ab60 2024-04-21T10:31:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f040fe76740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0408071cc0 con 0x561e6592b8d0 2024-04-21T10:31:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f040fe76740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f040806fb10 con 0x561e659340e0 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f0406ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592ab60 0x561e65950cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f0406ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592ab60 0x561e65950cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57744/0 (socket says 172.21.15.26:57744) 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f0406ffd640 1 -- 172.21.15.26:0/3444573994 learned_addr learned my addr 172.21.15.26:0/3444573994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f04077fe640 1 --2- 172.21.15.26:0/3444573994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e659340e0 0x561e659334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f03fffff640 1 --2- 172.21.15.26:0/3444573994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592b8d0 0x561e6592a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f0406ffd640 1 -- 172.21.15.26:0/3444573994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e659340e0 msgr2=0x561e659334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f0406ffd640 1 --2- 172.21.15.26:0/3444573994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e659340e0 0x561e659334f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f0406ffd640 1 -- 172.21.15.26:0/3444573994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592b8d0 msgr2=0x561e6592a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f0406ffd640 1 --2- 172.21.15.26:0/3444573994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592b8d0 0x561e6592a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f0406ffd640 1 -- 172.21.15.26:0/3444573994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e65929e30 con 0x561e6592ab60 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.189+0000 7f03fffff640 1 --2- 172.21.15.26:0/3444573994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592b8d0 0x561e6592a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.190+0000 7f0406ffd640 1 --2- 172.21.15.26:0/3444573994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592ab60 0x561e65950cf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f03f800bc30 tx=0x7f03f8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2144dc65dc1501 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.190+0000 7f03fcff9640 1 -- 172.21.15.26:0/3444573994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f800b770 con 0x561e6592ab60 2024-04-21T10:31:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.190+0000 7f03fcff9640 1 -- 172.21.15.26:0/3444573994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f8029070 con 0x561e6592ab60 2024-04-21T10:31:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.190+0000 7f03fcff9640 1 -- 172.21.15.26:0/3444573994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f802a460 con 0x561e6592ab60 2024-04-21T10:31:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.191+0000 7f040fe76740 1 -- 172.21.15.26:0/3444573994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592ab60 msgr2=0x561e65950cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.191+0000 7f040fe76740 1 --2- 172.21.15.26:0/3444573994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592ab60 0x561e65950cf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f03f800bc30 tx=0x7f03f8005e50 comp rx=0 tx=0).stop 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.191+0000 7f040fe76740 1 -- 172.21.15.26:0/3444573994 shutdown_connections 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.191+0000 7f040fe76740 1 --2- 172.21.15.26:0/3444573994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e659340e0 0x561e659334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.191+0000 7f040fe76740 1 --2- 172.21.15.26:0/3444573994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592b8d0 0x561e6592a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.191+0000 7f040fe76740 1 --2- 172.21.15.26:0/3444573994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e6592ab60 0x561e65950cf0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.191+0000 7f040fe76740 1 -- 172.21.15.26:0/3444573994 >> 172.21.15.26:0/3444573994 conn(0x561e658f1c90 msgr2=0x561e6592f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.191+0000 7f040fe76740 1 -- 172.21.15.26:0/3444573994 shutdown_connections 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.191+0000 7f040fe76740 1 -- 172.21.15.26:0/3444573994 wait complete. 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.191+0000 7f040fe76740 1 -- start start 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.191+0000 7f040fe76740 1 Processor -- start 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f040fe76740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e6594d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f040fe76740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e6592b8d0 0x561e65934ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f040fe76740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 0x561e65935000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f040fe76740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0408070180 con 0x561e659340e0 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f040fe76740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04080715f0 con 0x561e6592ab60 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f040fe76740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0408070850 con 0x561e6592b8d0 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f040fe76740 10 client.0 _ll_register_callbacks cb 0x7f0400005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f03fffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e6592b8d0 0x561e65934ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f03fffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e6592b8d0 0x561e65934ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.26:46950/0 (socket says 172.21.15.26:46950) 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f03fffff640 1 -- 172.21.15.26:0/2634796267 learned_addr learned my addr 172.21.15.26:0/2634796267 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f04077fe640 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 0x561e65935000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f0406ffd640 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e6594d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f03fffff640 1 -- 172.21.15.26:0/2634796267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 msgr2=0x561e6594d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f03fffff640 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e6594d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f03fffff640 1 -- 172.21.15.26:0/2634796267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 msgr2=0x561e65935000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f03fffff640 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 0x561e65935000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f03fffff640 1 -- 172.21.15.26:0/2634796267 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e659357e0 con 0x561e6592b8d0 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.192+0000 7f04077fe640 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 0x561e65935000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f03fffff640 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e6592b8d0 0x561e65934ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x561e6592b600 tx=0x7f03f0014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c58465cc58602c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f03ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f03ee7fc640 1 -- 172.21.15.26:0/2634796267 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f002b060 con 0x561e6592b8d0 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f040fe76740 1 -- 172.21.15.26:0/2634796267 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f8029070 con 0x561e6592b8d0 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f03ee7fc640 1 -- 172.21.15.26:0/2634796267 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f03f0010d30 con 0x561e6592b8d0 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f03ee7fc640 1 -- 172.21.15.26:0/2634796267 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f03f001b460 con 0x561e6592b8d0 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f040fe76740 10 client.25237 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f040fe76740 1 -- 172.21.15.26:0/2634796267 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561e659bb6f0 con 0x561e6592b8d0 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f03ee7fc640 1 -- 172.21.15.26:0/2634796267 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f8029070 con 0x561e6592b8d0 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f03ee7fc640 1 -- 172.21.15.26:0/2634796267 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f03f0019040 con 0x561e6592b8d0 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f040fe76740 10 client.25237 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f040fe76740 10 client.25237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f040fe76740 20 client.25237 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f040fe76740 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x561e659bbe50 0x561e659be270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f040fe76740 4 client.25237 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T10:31:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.193+0000 7f040fe76740 1 -- 172.21.15.26:0/2634796267 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e6582f3d0 con 0x561e659bbe50 2024-04-21T10:31:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.194+0000 7f0406ffd640 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x561e659bbe50 0x561e659be270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.194+0000 7f0406ffd640 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x561e659bbe50 0x561e659be270 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2dcaace9f6e061 server_cookie=5ef2979c95b69260 in_seq=0 out_seq=0 2024-04-21T10:31:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.194+0000 7f03ee7fc640 10 client.25237 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.198+0000 7f03ee7fc640 1 -- 172.21.15.26:0/2634796267 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e6582f3d0 con 0x561e659bbe50 2024-04-21T10:31:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.198+0000 7f03ee7fc640 10 client.25237 handle_command_reply: tid=0 2024-04-21T10:31:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.198+0000 7f040fe76740 1 client.25237 shutdown 2024-04-21T10:31:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.198+0000 7f040fe76740 1 -- 172.21.15.26:0/2634796267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e6592b8d0 msgr2=0x561e65934ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.198+0000 7f040fe76740 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e6592b8d0 0x561e65934ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x561e6592b600 tx=0x7f03f0014040 comp rx=0 tx=0).stop 2024-04-21T10:31:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.198+0000 7f040fe76740 1 -- 172.21.15.26:0/2634796267 shutdown_connections 2024-04-21T10:31:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.198+0000 7f040fe76740 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x561e659bbe50 0x561e659be270 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.198+0000 7f03ee7fc640 0 client.25237 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.198+0000 7f040fe76740 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e659340e0 0x561e65935000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.198+0000 7f040fe76740 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e6592b8d0 0x561e65934ac0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x561e6592b600 tx=0x7f03f0014040 comp rx=0 tx=0).stop 2024-04-21T10:31:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.198+0000 7f040fe76740 1 --2- 172.21.15.26:0/2634796267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561e6592ab60 0x561e6594d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.198+0000 7f040fe76740 1 -- 172.21.15.26:0/2634796267 >> 172.21.15.26:0/2634796267 conn(0x561e658f1c90 msgr2=0x561e65926ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.199+0000 7f040fe76740 1 -- 172.21.15.26:0/2634796267 shutdown_connections 2024-04-21T10:31:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.199+0000 7f040fe76740 1 -- 172.21.15.26:0/2634796267 wait complete. 2024-04-21T10:31:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.199+0000 7f040fe76740 20 client.25237 trim_cache size 0 max 16384 2024-04-21T10:31:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-04-21T10:31:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-04-21T10:31:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-04-21T10:31:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-04-21T10:31:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.199+0000 7f040d281640 1 -- 172.21.15.26:0/3235355618 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f03f40a3ff0 msgr2=0x7f03f40a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.199+0000 7f040d281640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f03f40a3ff0 0x7f03f40a64b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x561e659269e0 tx=0x7f03f000b580 comp rx=0 tx=0).stop 2024-04-21T10:31:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.199+0000 7f040d281640 1 -- 172.21.15.26:0/3235355618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0408199f50 msgr2=0x7f04081a4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.199+0000 7f040d281640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0408199f50 0x7f04081a4500 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0400005e30 tx=0x7f0400034670 comp rx=0 tx=0).stop 2024-04-21T10:31:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.200+0000 7f040d281640 1 -- 172.21.15.26:0/3235355618 shutdown_connections 2024-04-21T10:31:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.200+0000 7f040d281640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f03f40a3ff0 0x7f03f40a64b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.200+0000 7f040d281640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0408199f50 0x7f04081a4500 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.200+0000 7f040d281640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0408106170 0x7f04081999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.200+0000 7f040d281640 1 --2- 172.21.15.26:0/3235355618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04081010f0 0x7f0408199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.200+0000 7f040d281640 1 -- 172.21.15.26:0/3235355618 >> 172.21.15.26:0/3235355618 conn(0x7f0408077fd0 msgr2=0x7f04080fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.200+0000 7f040d281640 1 -- 172.21.15.26:0/3235355618 shutdown_connections 2024-04-21T10:31:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:40.200+0000 7f040d281640 1 -- 172.21.15.26:0/3235355618 wait complete. 2024-04-21T10:31:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:41 smithi026 ceph-mon[27965]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-21T10:31:41.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:41 smithi169 ceph-mon[31469]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-21T10:31:41.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:41 smithi190 ceph-mon[31423]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-21T10:31:42.068 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:31:42.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.173+0000 7f8644fb2640 1 -- 172.21.15.26:0/477536756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8640074320 msgr2=0x7f8640074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.173+0000 7f8644fb2640 1 --2- 172.21.15.26:0/477536756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8640074320 0x7f8640074700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8630009900 tx=0x7f863002bcd0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.173+0000 7f8644fb2640 1 -- 172.21.15.26:0/477536756 shutdown_connections 2024-04-21T10:31:42.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.173+0000 7f8644fb2640 1 --2- 172.21.15.26:0/477536756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864010e7b0 0x7f8640110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.173+0000 7f8644fb2640 1 --2- 172.21.15.26:0/477536756 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8640074c40 0x7f864010e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.173+0000 7f8644fb2640 1 --2- 172.21.15.26:0/477536756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8640074320 0x7f8640074700 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.173+0000 7f8644fb2640 1 -- 172.21.15.26:0/477536756 >> 172.21.15.26:0/477536756 conn(0x7f8640079350 msgr2=0x7f8640079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:42.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.173+0000 7f8644fb2640 1 -- 172.21.15.26:0/477536756 shutdown_connections 2024-04-21T10:31:42.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.173+0000 7f8644fb2640 1 -- 172.21.15.26:0/477536756 wait complete. 2024-04-21T10:31:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.174+0000 7f8644fb2640 1 Processor -- start 2024-04-21T10:31:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.174+0000 7f8644fb2640 1 -- start start 2024-04-21T10:31:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.174+0000 7f8644fb2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8640074320 0x7f86401a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.174+0000 7f8644fb2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8640074c40 0x7f86401a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.174+0000 7f8644fb2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864010e7b0 0x7f864019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.174+0000 7f8644fb2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f86401143f0 con 0x7f8640074320 2024-04-21T10:31:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.174+0000 7f8644fb2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8640114270 con 0x7f8640074c40 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.174+0000 7f8644fb2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8640114570 con 0x7f864010e7b0 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f863e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8640074320 0x7f86401a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f863e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8640074320 0x7f86401a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57788/0 (socket says 172.21.15.26:57788) 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f863e59c640 1 -- 172.21.15.26:0/1763723483 learned_addr learned my addr 172.21.15.26:0/1763723483 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f863dd9b640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8640074c40 0x7f86401a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f863ed9d640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864010e7b0 0x7f864019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f863e59c640 1 -- 172.21.15.26:0/1763723483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864010e7b0 msgr2=0x7f864019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f863e59c640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864010e7b0 0x7f864019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f863e59c640 1 -- 172.21.15.26:0/1763723483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8640074c40 msgr2=0x7f86401a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f863e59c640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8640074c40 0x7f86401a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f863e59c640 1 -- 172.21.15.26:0/1763723483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864019afa0 con 0x7f8640074320 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f863dd9b640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8640074c40 0x7f86401a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f863e59c640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8640074320 0x7f86401a0660 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8630005940 tx=0x7f86300045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.175+0000 7f862f7fe640 1 -- 172.21.15.26:0/1763723483 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630043020 con 0x7f8640074320 2024-04-21T10:31:42.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.176+0000 7f862f7fe640 1 -- 172.21.15.26:0/1763723483 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f863002dba0 con 0x7f8640074320 2024-04-21T10:31:42.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.176+0000 7f8644fb2640 1 -- 172.21.15.26:0/1763723483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864019b230 con 0x7f8640074320 2024-04-21T10:31:42.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.176+0000 7f862f7fe640 1 -- 172.21.15.26:0/1763723483 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630003930 con 0x7f8640074320 2024-04-21T10:31:42.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.176+0000 7f8644fb2640 1 -- 172.21.15.26:0/1763723483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86401a74b0 con 0x7f8640074320 2024-04-21T10:31:42.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.177+0000 7f8644fb2640 1 -- 172.21.15.26:0/1763723483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86400759d0 con 0x7f8640074320 2024-04-21T10:31:42.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.179+0000 7f862f7fe640 1 -- 172.21.15.26:0/1763723483 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f8630046020 con 0x7f8640074320 2024-04-21T10:31:42.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.180+0000 7f862f7fe640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f86280a3f50 0x7f86280a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.180+0000 7f862f7fe640 1 -- 172.21.15.26:0/1763723483 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f86300f29c0 con 0x7f8640074320 2024-04-21T10:31:42.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.180+0000 7f863dd9b640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f86280a3f50 0x7f86280a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.181+0000 7f863dd9b640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f86280a3f50 0x7f86280a6410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8640110770 tx=0x7f8634005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:42.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.189+0000 7f862f7fe640 1 -- 172.21.15.26:0/1763723483 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f863004d020 con 0x7f8640074320 2024-04-21T10:31:42.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.651+0000 7f8644fb2640 1 -- 172.21.15.26:0/1763723483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8640109930 con 0x7f8640074320 2024-04-21T10:31:42.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.652+0000 7f862f7fe640 1 -- 172.21.15.26:0/1763723483 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7f86300bef40 con 0x7f8640074320 2024-04-21T10:31:42.652 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:42.652 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:30:56.186598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24478,24488]},"id":1}]} 2024-04-21T10:31:42.652 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 21 2024-04-21T10:31:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.655+0000 7f8644fb2640 1 -- 172.21.15.26:0/1763723483 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f86280a3f50 msgr2=0x7f86280a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.655+0000 7f8644fb2640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f86280a3f50 0x7f86280a6410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8640110770 tx=0x7f8634005f90 comp rx=0 tx=0).stop 2024-04-21T10:31:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.655+0000 7f8644fb2640 1 -- 172.21.15.26:0/1763723483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8640074320 msgr2=0x7f86401a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.655+0000 7f8644fb2640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8640074320 0x7f86401a0660 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8630005940 tx=0x7f86300045e0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.655+0000 7f8644fb2640 1 -- 172.21.15.26:0/1763723483 shutdown_connections 2024-04-21T10:31:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.655+0000 7f8644fb2640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f86280a3f50 0x7f86280a6410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.655+0000 7f8644fb2640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864010e7b0 0x7f864019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.655+0000 7f8644fb2640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8640074c40 0x7f86401a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.655+0000 7f8644fb2640 1 --2- 172.21.15.26:0/1763723483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8640074320 0x7f86401a0660 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.655+0000 7f8644fb2640 1 -- 172.21.15.26:0/1763723483 >> 172.21.15.26:0/1763723483 conn(0x7f8640079350 msgr2=0x7f86400fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.655+0000 7f8644fb2640 1 -- 172.21.15.26:0/1763723483 shutdown_connections 2024-04-21T10:31:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.655+0000 7f8644fb2640 1 -- 172.21.15.26:0/1763723483 wait complete. 2024-04-21T10:31:42.671 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id 9e02290a --release --await' 2024-04-21T10:31:42.671 DEBUG:teuthology.orchestra.run.smithi026:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24496 quiesce db --set-id 9e02290a --release --await 2024-04-21T10:31:42.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.832+0000 7f8e24274640 1 -- 172.21.15.26:0/3142385675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e1c103af0 msgr2=0x7f8e1c10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.832+0000 7f8e24274640 1 --2- 172.21.15.26:0/3142385675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e1c103af0 0x7f8e1c10bfd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009900 tx=0x7f8e0c02bd60 comp rx=0 tx=0).stop 2024-04-21T10:31:42.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.832+0000 7f8e24274640 1 -- 172.21.15.26:0/3142385675 shutdown_connections 2024-04-21T10:31:42.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.832+0000 7f8e24274640 1 --2- 172.21.15.26:0/3142385675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e1c10c510 0x7f8e1c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.832+0000 7f8e24274640 1 --2- 172.21.15.26:0/3142385675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e1c103af0 0x7f8e1c10bfd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.832+0000 7f8e24274640 1 --2- 172.21.15.26:0/3142385675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1c1031d0 0x7f8e1c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.832+0000 7f8e24274640 1 -- 172.21.15.26:0/3142385675 >> 172.21.15.26:0/3142385675 conn(0x7f8e1c077f50 msgr2=0x7f8e1c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:42.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.832+0000 7f8e24274640 1 -- 172.21.15.26:0/3142385675 shutdown_connections 2024-04-21T10:31:42.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.832+0000 7f8e24274640 1 -- 172.21.15.26:0/3142385675 wait complete. 2024-04-21T10:31:42.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.833+0000 7f8e24274640 1 Processor -- start 2024-04-21T10:31:42.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.833+0000 7f8e24274640 1 -- start start 2024-04-21T10:31:42.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.833+0000 7f8e24274640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e1c1031d0 0x7f8e1c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.833+0000 7f8e24274640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e1c103af0 0x7f8e1c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.833+0000 7f8e2180f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e1c103af0 0x7f8e1c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.833+0000 7f8e2180f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e1c103af0 0x7f8e1c198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57812/0 (socket says 172.21.15.26:57812) 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.833+0000 7f8e2180f640 1 -- 172.21.15.26:0/2065484284 learned_addr learned my addr 172.21.15.26:0/2065484284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.833+0000 7f8e22010640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e1c1031d0 0x7f8e1c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.833+0000 7f8e24274640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1c10c510 0x7f8e1c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.833+0000 7f8e24274640 1 -- 172.21.15.26:0/2065484284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c112180 con 0x7f8e1c103af0 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.833+0000 7f8e24274640 1 -- 172.21.15.26:0/2065484284 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c112000 con 0x7f8e1c1031d0 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.833+0000 7f8e24274640 1 -- 172.21.15.26:0/2065484284 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c112300 con 0x7f8e1c10c510 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e2180f640 1 -- 172.21.15.26:0/2065484284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1c10c510 msgr2=0x7f8e1c19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e22811640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1c10c510 0x7f8e1c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1c10c510 0x7f8e1c19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e2180f640 1 -- 172.21.15.26:0/2065484284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e1c1031d0 msgr2=0x7f8e1c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e1c1031d0 0x7f8e1c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e2180f640 1 -- 172.21.15.26:0/2065484284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c19d690 con 0x7f8e1c103af0 2024-04-21T10:31:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e22811640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1c10c510 0x7f8e1c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:42.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e22010640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e1c1031d0 0x7f8e1c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:42.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e1c103af0 0x7f8e1c198be0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c005940 tx=0x7f8e0c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:42.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e137fe640 1 -- 172.21.15.26:0/2065484284 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c043020 con 0x7f8e1c103af0 2024-04-21T10:31:42.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e137fe640 1 -- 172.21.15.26:0/2065484284 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e0c02dc30 con 0x7f8e1c103af0 2024-04-21T10:31:42.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e137fe640 1 -- 172.21.15.26:0/2065484284 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0038e0 con 0x7f8e1c103af0 2024-04-21T10:31:42.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e24274640 1 -- 172.21.15.26:0/2065484284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1c19d920 con 0x7f8e1c103af0 2024-04-21T10:31:42.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.834+0000 7f8e24274640 1 -- 172.21.15.26:0/2065484284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e1c100d50 con 0x7f8e1c103af0 2024-04-21T10:31:42.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.838+0000 7f8e26e6b740 1 Processor -- start 2024-04-21T10:31:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.838+0000 7f8e26e6b740 1 -- start start 2024-04-21T10:31:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.838+0000 7f8e137fe640 1 -- 172.21.15.26:0/2065484284 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f8e0c046020 con 0x7f8e1c103af0 2024-04-21T10:31:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.838+0000 7f8e26e6b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2310 0x55bfe24a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.838+0000 7f8e26e6b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 0x55bfe24ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.838+0000 7f8e26e6b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 0x55bfe24adc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.838+0000 7f8e26e6b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bfe2407500 con 0x55bfe24ab810 2024-04-21T10:31:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.838+0000 7f8e26e6b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bfe23ad6a0 con 0x55bfe24a2310 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.838+0000 7f8e26e6b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bfe24034c0 con 0x55bfe24a2c30 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.838+0000 7f8e2180f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 0x55bfe24adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.838+0000 7f8e22010640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 0x55bfe24ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e22010640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 0x55bfe24ab2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.26:47002/0 (socket says 172.21.15.26:47002) 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e22010640 1 -- 172.21.15.26:0/2971100550 learned_addr learned my addr 172.21.15.26:0/2971100550 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e2180f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 0x55bfe24adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57818/0 (socket says 172.21.15.26:57818) 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e22811640 1 --2- 172.21.15.26:0/2971100550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2310 0x55bfe24a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e22010640 1 -- 172.21.15.26:0/2971100550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2310 msgr2=0x55bfe24a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e22010640 1 --2- 172.21.15.26:0/2971100550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2310 0x55bfe24a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e22010640 1 -- 172.21.15.26:0/2971100550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 msgr2=0x55bfe24adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e22010640 1 --2- 172.21.15.26:0/2971100550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 0x55bfe24adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e137fe640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f8e080a3f50 0x7f8e080a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e22010640 1 -- 172.21.15.26:0/2971100550 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe23cc5e0 con 0x55bfe24a2c30 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2971100550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 0x55bfe24adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:42.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e137fe640 1 -- 172.21.15.26:0/2065484284 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f8e0c0f2aa0 con 0x7f8e1c103af0 2024-04-21T10:31:42.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e22811640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f8e080a3f50 0x7f8e080a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.839+0000 7f8e22010640 1 --2- 172.21.15.26:0/2971100550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 0x55bfe24ab2d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800f4c0 tx=0x7f8e1800f980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad2d10ce9ea34604 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:42.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.840+0000 7f8e10bfe640 1 -- 172.21.15.26:0/2971100550 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18005220 con 0x55bfe24a2c30 2024-04-21T10:31:42.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.840+0000 7f8e10bfe640 1 -- 172.21.15.26:0/2971100550 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e180047a0 con 0x55bfe24a2c30 2024-04-21T10:31:42.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.840+0000 7f8e10bfe640 1 -- 172.21.15.26:0/2971100550 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18010b60 con 0x55bfe24a2c30 2024-04-21T10:31:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.841+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2971100550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 msgr2=0x55bfe24ab2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.841+0000 7f8e22811640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f8e080a3f50 0x7f8e080a6410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e14005fd0 tx=0x7f8e14009500 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.841+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2971100550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 0x55bfe24ab2d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800f4c0 tx=0x7f8e1800f980 comp rx=0 tx=0).stop 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.841+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2971100550 shutdown_connections 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.841+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2971100550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 0x55bfe24adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.841+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2971100550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 0x55bfe24ab2d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.841+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2971100550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2310 0x55bfe24a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.841+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2971100550 >> 172.21.15.26:0/2971100550 conn(0x55bfe249ca40 msgr2=0x55bfe249ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.841+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2971100550 shutdown_connections 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.841+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2971100550 wait complete. 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.841+0000 7f8e26e6b740 1 -- start start 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e26e6b740 1 Processor -- start 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e26e6b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2310 0x55bfe24ab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e26e6b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2c30 0x55bfe24a85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e26e6b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24ab810 0x55bfe24a8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e26e6b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bfe24ae6a0 con 0x55bfe24a2c30 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e26e6b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bfe23c0a60 con 0x55bfe24a2310 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e26e6b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55bfe24ae820 con 0x55bfe24ab810 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e26e6b740 10 client.0 _ll_register_callbacks cb 0x55bfe2119110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e2180f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2c30 0x55bfe24a85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e2180f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2c30 0x55bfe24a85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57832/0 (socket says 172.21.15.26:57832) 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e2180f640 1 -- 172.21.15.26:0/2340272846 learned_addr learned my addr 172.21.15.26:0/2340272846 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e22811640 1 --2- 172.21.15.26:0/2340272846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24ab810 0x55bfe24a8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e2180f640 1 -- 172.21.15.26:0/2340272846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24ab810 msgr2=0x55bfe24a8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2340272846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24ab810 0x55bfe24a8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e2180f640 1 -- 172.21.15.26:0/2340272846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2310 msgr2=0x55bfe24ab210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2340272846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2310 0x55bfe24ab210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.842+0000 7f8e2180f640 1 -- 172.21.15.26:0/2340272846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe2119920 con 0x55bfe24a2c30 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.843+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2340272846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2c30 0x55bfe24a85a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0f01c0 tx=0x7f8e0c0be7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe87b1ab034d660 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.843+0000 7f8e05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.843+0000 7f8e05ffb640 1 -- 172.21.15.26:0/2340272846 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0be920 con 0x55bfe24a2c30 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.843+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2340272846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e180047a0 con 0x55bfe24a2c30 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.843+0000 7f8e05ffb640 1 -- 172.21.15.26:0/2340272846 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f8e0c0c7100 con 0x55bfe24a2c30 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.843+0000 7f8e05ffb640 1 -- 172.21.15.26:0/2340272846 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8e0c0c2cd0 con 0x55bfe24a2c30 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.843+0000 7f8e26e6b740 10 client.15660 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.843+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2340272846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bfe21e5d30 con 0x55bfe24a2c30 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.844+0000 7f8e05ffb640 1 -- 172.21.15.26:0/2340272846 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e180047a0 con 0x55bfe24a2c30 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.844+0000 7f8e05ffb640 1 -- 172.21.15.26:0/2340272846 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8e0c0ddc70 con 0x55bfe24a2c30 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.844+0000 7f8e26e6b740 10 client.15660 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.844+0000 7f8e26e6b740 10 client.15660 resolve_mds: validated gid 24496 aka daemon mds.i 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.844+0000 7f8e26e6b740 20 client.15660 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.844+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2340272846 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55bfe24bbb00 0x55bfe24bdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.844+0000 7f8e26e6b740 4 client.15660 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.844+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2340272846 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfe23daa10 con 0x55bfe24bbb00 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.845+0000 7f8e22010640 1 --2- 172.21.15.26:0/2340272846 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55bfe24bbb00 0x55bfe24bdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.845+0000 7f8e05ffb640 10 client.15660 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.845+0000 7f8e22010640 1 --2- 172.21.15.26:0/2340272846 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55bfe24bbb00 0x55bfe24bdf80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd47d55278f075b server_cookie=43e88a88201cef91 in_seq=0 out_seq=0 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.846+0000 7f8e05ffb640 1 -- 172.21.15.26:0/2340272846 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bfe23daa10 con 0x55bfe24bbb00 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.846+0000 7f8e05ffb640 10 client.15660 handle_command_reply: tid=0 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.846+0000 7f8e26e6b740 1 client.15660 shutdown 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.846+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2340272846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2c30 msgr2=0x55bfe24a85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.846+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2340272846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2c30 0x55bfe24a85a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0f01c0 tx=0x7f8e0c0be7b0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.846+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2340272846 shutdown_connections 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.846+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2340272846 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55bfe24bbb00 0x55bfe24bdf80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.846+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2340272846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24ab810 0x55bfe24a8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.846+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2340272846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2c30 0x55bfe24a85a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.846+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2340272846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2310 0x55bfe24ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.846+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2340272846 >> 172.21.15.26:0/2340272846 conn(0x55bfe249ca40 msgr2=0x55bfe249e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.847+0000 7f8e05ffb640 0 client.15660 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.847+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2340272846 shutdown_connections 2024-04-21T10:31:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.847+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2340272846 wait complete. 2024-04-21T10:31:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.847+0000 7f8e26e6b740 20 client.15660 trim_cache size 0 max 16384 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.864+0000 7f8e26e6b740 1 Processor -- start 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.864+0000 7f8e26e6b740 1 -- start start 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.864+0000 7f8e26e6b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2310 0x55bfe24b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.864+0000 7f8e26e6b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2c30 0x55bfe24c3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.864+0000 7f8e26e6b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 0x55bfe24c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.864+0000 7f8e26e6b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1b2620 con 0x55bfe24ab810 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.864+0000 7f8e26e6b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1afef0 con 0x55bfe24a2c30 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.864+0000 7f8e26e6b740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1add40 con 0x55bfe24a2310 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.864+0000 7f8e22811640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 0x55bfe24c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e22811640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 0x55bfe24c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57836/0 (socket says 172.21.15.26:57836) 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e22811640 1 -- 172.21.15.26:0/2727442016 learned_addr learned my addr 172.21.15.26:0/2727442016 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2727442016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2c30 0x55bfe24c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e22010640 1 --2- 172.21.15.26:0/2727442016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2310 0x55bfe24b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e22811640 1 -- 172.21.15.26:0/2727442016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2310 msgr2=0x55bfe24b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e22811640 1 --2- 172.21.15.26:0/2727442016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2310 0x55bfe24b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e22811640 1 -- 172.21.15.26:0/2727442016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2c30 msgr2=0x55bfe24c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e22811640 1 --2- 172.21.15.26:0/2727442016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2c30 0x55bfe24c3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e22811640 1 -- 172.21.15.26:0/2727442016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe24a8f20 con 0x55bfe24ab810 2024-04-21T10:31:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2727442016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2c30 0x55bfe24c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e22811640 1 --2- 172.21.15.26:0/2727442016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 0x55bfe24c3dd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8e14010780 tx=0x7f8e14015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c87a6e1a99c423f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e07fff640 1 -- 172.21.15.26:0/2727442016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1402c070 con 0x55bfe24ab810 2024-04-21T10:31:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e07fff640 1 -- 172.21.15.26:0/2727442016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e14010e30 con 0x55bfe24ab810 2024-04-21T10:31:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.865+0000 7f8e07fff640 1 -- 172.21.15.26:0/2727442016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1401e870 con 0x55bfe24ab810 2024-04-21T10:31:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.866+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2727442016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 msgr2=0x55bfe24c3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.866+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2727442016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 0x55bfe24c3dd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8e14010780 tx=0x7f8e14015000 comp rx=0 tx=0).stop 2024-04-21T10:31:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.866+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2727442016 shutdown_connections 2024-04-21T10:31:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.866+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2727442016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24ab810 0x55bfe24c3dd0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.866+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2727442016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24a2c30 0x55bfe24c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.866+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2727442016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2310 0x55bfe24b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.866+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2727442016 >> 172.21.15.26:0/2727442016 conn(0x55bfe249ca40 msgr2=0x55bfe249da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.867+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2727442016 shutdown_connections 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.867+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2727442016 wait complete. 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.867+0000 7f8e26e6b740 1 -- start start 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.867+0000 7f8e26e6b740 1 Processor -- start 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.867+0000 7f8e26e6b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2310 0x55bfe24aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.867+0000 7f8e26e6b740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 0x55bfe24aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.867+0000 7f8e26e6b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24ab810 0x55bfe24af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.867+0000 7f8e26e6b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1ae3b0 con 0x55bfe24a2310 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e22010640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2310 0x55bfe24aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e22010640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2310 0x55bfe24aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57846/0 (socket says 172.21.15.26:57846) 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e22010640 1 -- 172.21.15.26:0/2053622350 learned_addr learned my addr 172.21.15.26:0/2053622350 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e26e6b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1af820 con 0x55bfe24ab810 2024-04-21T10:31:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2053622350 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1aea80 con 0x55bfe24a2c30 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 0x55bfe24aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e26e6b740 10 client.0 _ll_register_callbacks cb 0x7f8e0c0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e22010640 1 -- 172.21.15.26:0/2053622350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 msgr2=0x55bfe24aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e22010640 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 0x55bfe24aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e22811640 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24ab810 0x55bfe24af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e22010640 1 -- 172.21.15.26:0/2053622350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24ab810 msgr2=0x55bfe24af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e22010640 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24ab810 0x55bfe24af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e22010640 1 -- 172.21.15.26:0/2053622350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe2522200 con 0x55bfe24a2310 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 0x55bfe24aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e22811640 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24ab810 0x55bfe24af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e22010640 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2310 0x55bfe24aea30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800ac00 tx=0x7f8e18010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d13dd797228464 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.868+0000 7f8e05ffb640 1 -- 172.21.15.26:0/2053622350 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18004cb0 con 0x55bfe24a2310 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e05ffb640 1 -- 172.21.15.26:0/2053622350 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f8e18010870 con 0x55bfe24a2310 2024-04-21T10:31:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2053622350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e14010e30 con 0x55bfe24a2310 2024-04-21T10:31:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e05ffb640 1 -- 172.21.15.26:0/2053622350 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8e18004870 con 0x55bfe24a2310 2024-04-21T10:31:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e26e6b740 10 client.15666 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2053622350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bfe2523500 con 0x55bfe24a2310 2024-04-21T10:31:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e05ffb640 1 -- 172.21.15.26:0/2053622350 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e14010e30 con 0x55bfe24a2310 2024-04-21T10:31:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e05ffb640 1 -- 172.21.15.26:0/2053622350 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8e18018440 con 0x55bfe24a2310 2024-04-21T10:31:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e26e6b740 10 client.15666 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e26e6b740 10 client.15666 resolve_mds: validated gid 24496 aka daemon mds.i 2024-04-21T10:31:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e26e6b740 20 client.15666 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55bfe2523950 0x55bfe2525d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e26e6b740 4 client.15666 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9e02290a", "release": true, "await": true}] 2024-04-21T10:31:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.869+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2053622350 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9e02290a", "release": true, "await": true}) -- 0x55bfe2414f20 con 0x55bfe2523950 2024-04-21T10:31:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.870+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55bfe2523950 0x55bfe2525d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.870+0000 7f8e2180f640 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55bfe2523950 0x55bfe2525d70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92065394f29bc966 server_cookie=492801091e19393d in_seq=0 out_seq=0 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.870+0000 7f8e05ffb640 10 client.15666 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.872+0000 7f8e05ffb640 1 -- 172.21.15.26:0/2053622350 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55bfe2414f20 con 0x55bfe2523950 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.872+0000 7f8e05ffb640 10 client.15666 handle_command_reply: tid=0 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.872+0000 7f8e26e6b740 1 client.15666 shutdown 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.872+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2053622350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2310 msgr2=0x55bfe24aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.872+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2310 0x55bfe24aea30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800ac00 tx=0x7f8e18010800 comp rx=0 tx=0).stop 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.872+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2053622350 shutdown_connections 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.872+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55bfe2523950 0x55bfe2525d70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.872+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe24ab810 0x55bfe24af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.872+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bfe24a2c30 0x55bfe24aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.872+0000 7f8e26e6b740 1 --2- 172.21.15.26:0/2053622350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bfe24a2310 0x55bfe24aea30 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.872+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2053622350 >> 172.21.15.26:0/2053622350 conn(0x55bfe249ca40 msgr2=0x55bfe249e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.872+0000 7f8e05ffb640 0 client.15666 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2053622350 shutdown_connections 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e26e6b740 1 -- 172.21.15.26:0/2053622350 wait complete. 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e26e6b740 20 client.15666 trim_cache size 0 max 16384 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 21, 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24496, 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 4, 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stdout: "9e02290a": { 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 4, 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-04-21T10:31:42.875 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 30.0, 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 79.0, 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:42.876 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:31:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e24274640 1 -- 172.21.15.26:0/2065484284 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f8e080a3f50 msgr2=0x7f8e080a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e24274640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f8e080a3f50 0x7f8e080a6410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e14005fd0 tx=0x7f8e14009500 comp rx=0 tx=0).stop 2024-04-21T10:31:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e24274640 1 -- 172.21.15.26:0/2065484284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e1c103af0 msgr2=0x7f8e1c198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e24274640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e1c103af0 0x7f8e1c198be0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c005940 tx=0x7f8e0c0045e0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e24274640 1 -- 172.21.15.26:0/2065484284 shutdown_connections 2024-04-21T10:31:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e24274640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f8e080a3f50 0x7f8e080a6410 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e24274640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e1c10c510 0x7f8e1c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e24274640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e1c103af0 0x7f8e1c198be0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e24274640 1 --2- 172.21.15.26:0/2065484284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e1c1031d0 0x7f8e1c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.873+0000 7f8e24274640 1 -- 172.21.15.26:0/2065484284 >> 172.21.15.26:0/2065484284 conn(0x7f8e1c077f50 msgr2=0x7f8e1c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.874+0000 7f8e24274640 1 -- 172.21.15.26:0/2065484284 shutdown_connections 2024-04-21T10:31:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.874+0000 7f8e24274640 1 -- 172.21.15.26:0/2065484284 wait complete. 2024-04-21T10:31:42.887 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: 9e02290a, seconds elapsed: 20.3 2024-04-21T10:31:42.887 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.95 2024-04-21T10:31:42.887 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.997+0000 7f1abf06d640 1 -- 172.21.15.26:0/3197898367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8069620 msgr2=0x7f1ab8069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.997+0000 7f1abf06d640 1 --2- 172.21.15.26:0/3197898367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8069620 0x7f1ab8069aa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009900 tx=0x7f1aa402bc30 comp rx=0 tx=0).stop 2024-04-21T10:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.997+0000 7f1abf06d640 1 -- 172.21.15.26:0/3197898367 shutdown_connections 2024-04-21T10:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.997+0000 7f1abf06d640 1 --2- 172.21.15.26:0/3197898367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ab810e810 0x7f1ab8110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.997+0000 7f1abf06d640 1 --2- 172.21.15.26:0/3197898367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8069620 0x7f1ab8069aa0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.997+0000 7f1abf06d640 1 --2- 172.21.15.26:0/3197898367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab8102e30 0x7f1ab80690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.997+0000 7f1abf06d640 1 -- 172.21.15.26:0/3197898367 >> 172.21.15.26:0/3197898367 conn(0x7f1ab8077e30 msgr2=0x7f1ab8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.997+0000 7f1abf06d640 1 -- 172.21.15.26:0/3197898367 shutdown_connections 2024-04-21T10:31:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.997+0000 7f1abf06d640 1 -- 172.21.15.26:0/3197898367 wait complete. 2024-04-21T10:31:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1abf06d640 1 Processor -- start 2024-04-21T10:31:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1abf06d640 1 -- start start 2024-04-21T10:31:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1abf06d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ab8069620 0x7f1ab81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1abf06d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8102e30 0x7f1ab81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1abf06d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab810e810 0x7f1ab819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1abf06d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81143e0 con 0x7f1ab8102e30 2024-04-21T10:31:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1abf06d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8114260 con 0x7f1ab810e810 2024-04-21T10:31:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1abf06d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8114560 con 0x7f1ab8069620 2024-04-21T10:31:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1ab7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8102e30 0x7f1ab81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1ab7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8102e30 0x7f1ab81a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57852/0 (socket says 172.21.15.26:57852) 2024-04-21T10:31:42.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1ab7fff640 1 -- 172.21.15.26:0/4267482641 learned_addr learned my addr 172.21.15.26:0/4267482641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1abce09640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ab8069620 0x7f1ab81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1abd60a640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab810e810 0x7f1ab819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1ab7fff640 1 -- 172.21.15.26:0/4267482641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ab8069620 msgr2=0x7f1ab81a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1ab7fff640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ab8069620 0x7f1ab81a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1ab7fff640 1 -- 172.21.15.26:0/4267482641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab810e810 msgr2=0x7f1ab819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1ab7fff640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab810e810 0x7f1ab819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1ab7fff640 1 -- 172.21.15.26:0/4267482641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab819afd0 con 0x7f1ab8102e30 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1abd60a640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab810e810 0x7f1ab819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.998+0000 7f1abce09640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ab8069620 0x7f1ab81a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.999+0000 7f1ab7fff640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8102e30 0x7f1ab81a0bd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4000c00 tx=0x7f1aa402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.999+0000 7f1ab5ffb640 1 -- 172.21.15.26:0/4267482641 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4043020 con 0x7f1ab8102e30 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.999+0000 7f1ab5ffb640 1 -- 172.21.15.26:0/4267482641 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa402cbb0 con 0x7f1ab8102e30 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.999+0000 7f1ab5ffb640 1 -- 172.21.15.26:0/4267482641 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4003ac0 con 0x7f1ab8102e30 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.999+0000 7f1abf06d640 1 -- 172.21.15.26:0/4267482641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab819b260 con 0x7f1ab8102e30 2024-04-21T10:31:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:42.999+0000 7f1abf06d640 1 -- 172.21.15.26:0/4267482641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab81a74e0 con 0x7f1ab8102e30 2024-04-21T10:31:43.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.000+0000 7f1abf06d640 1 -- 172.21.15.26:0/4267482641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab8101430 con 0x7f1ab8102e30 2024-04-21T10:31:43.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.002+0000 7f1ab5ffb640 1 -- 172.21.15.26:0/4267482641 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f1aa4046020 con 0x7f1ab8102e30 2024-04-21T10:31:43.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.003+0000 7f1ab5ffb640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1aa80a3f50 0x7f1aa80a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.003+0000 7f1abce09640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1aa80a3f50 0x7f1aa80a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.003+0000 7f1ab5ffb640 1 -- 172.21.15.26:0/4267482641 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f1aa40f2c90 con 0x7f1ab8102e30 2024-04-21T10:31:43.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.003+0000 7f1abce09640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1aa80a3f50 0x7f1aa80a6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00bfd0 tx=0x7f1aac00b390 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:43.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.007+0000 7f1ab5ffb640 1 -- 172.21.15.26:0/4267482641 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1aa404d070 con 0x7f1ab8102e30 2024-04-21T10:31:43.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.235+0000 7f1abf06d640 1 -- 172.21.15.26:0/4267482641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ab8068de0 con 0x7f1ab8102e30 2024-04-21T10:31:43.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.235+0000 7f1ab5ffb640 1 -- 172.21.15.26:0/4267482641 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7f1aa40bf210 con 0x7f1ab8102e30 2024-04-21T10:31:43.235 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:31:43.235 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:30:56.186598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24478,24488]},"id":1}]} 2024-04-21T10:31:43.235 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 21 2024-04-21T10:31:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.237+0000 7f1abf06d640 1 -- 172.21.15.26:0/4267482641 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1aa80a3f50 msgr2=0x7f1aa80a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.237+0000 7f1abf06d640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1aa80a3f50 0x7f1aa80a6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00bfd0 tx=0x7f1aac00b390 comp rx=0 tx=0).stop 2024-04-21T10:31:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.237+0000 7f1abf06d640 1 -- 172.21.15.26:0/4267482641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8102e30 msgr2=0x7f1ab81a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.237+0000 7f1abf06d640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8102e30 0x7f1ab81a0bd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4000c00 tx=0x7f1aa402c6c0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.238+0000 7f1abf06d640 1 -- 172.21.15.26:0/4267482641 shutdown_connections 2024-04-21T10:31:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.238+0000 7f1abf06d640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f1aa80a3f50 0x7f1aa80a6410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.238+0000 7f1abf06d640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab810e810 0x7f1ab819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.238+0000 7f1abf06d640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ab8102e30 0x7f1ab81a0bd0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.238+0000 7f1abf06d640 1 --2- 172.21.15.26:0/4267482641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ab8069620 0x7f1ab81a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.238+0000 7f1abf06d640 1 -- 172.21.15.26:0/4267482641 >> 172.21.15.26:0/4267482641 conn(0x7f1ab8077e30 msgr2=0x7f1ab810f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.238+0000 7f1abf06d640 1 -- 172.21.15.26:0/4267482641 shutdown_connections 2024-04-21T10:31:43.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.238+0000 7f1abf06d640 1 -- 172.21.15.26:0/4267482641 wait complete. 2024-04-21T10:31:43.248 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db / --timeout 30 --expiration 78.95 --await' 2024-04-21T10:31:43.249 DEBUG:teuthology.orchestra.run.smithi026:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24496 quiesce db / --timeout 30 --expiration 78.95 --await 2024-04-21T10:31:43.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.364+0000 7fc917043640 1 -- 172.21.15.26:0/4117142804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc910107260 msgr2=0x7fc91010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.364+0000 7fc917043640 1 --2- 172.21.15.26:0/4117142804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc910107260 0x7fc91010ee30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc908009ca0 tx=0x7fc90802c0f0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.364+0000 7fc917043640 1 -- 172.21.15.26:0/4117142804 shutdown_connections 2024-04-21T10:31:43.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.364+0000 7fc917043640 1 --2- 172.21.15.26:0/4117142804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc910107260 0x7fc91010ee30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.364+0000 7fc917043640 1 --2- 172.21.15.26:0/4117142804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc910102b40 0x7fc910106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.364+0000 7fc917043640 1 --2- 172.21.15.26:0/4117142804 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc910102190 0x7fc910102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.364+0000 7fc917043640 1 -- 172.21.15.26:0/4117142804 >> 172.21.15.26:0/4117142804 conn(0x7fc910079310 msgr2=0x7fc910079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:43.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.364+0000 7fc917043640 1 -- 172.21.15.26:0/4117142804 shutdown_connections 2024-04-21T10:31:43.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.364+0000 7fc917043640 1 -- 172.21.15.26:0/4117142804 wait complete. 2024-04-21T10:31:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.365+0000 7fc917043640 1 Processor -- start 2024-04-21T10:31:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.365+0000 7fc917043640 1 -- start start 2024-04-21T10:31:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.365+0000 7fc917043640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc910102190 0x7fc910074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.365+0000 7fc917043640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc910107260 0x7fc910074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.365+0000 7fc917043640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc910074cd0 0x7fc9101a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.365+0000 7fc917043640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101121b0 con 0x7fc910102190 2024-04-21T10:31:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.365+0000 7fc917043640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc910112030 con 0x7fc910074cd0 2024-04-21T10:31:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.365+0000 7fc917043640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc910112330 con 0x7fc910107260 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.365+0000 7fc914ddf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc910102190 0x7fc910074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc914ddf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc910102190 0x7fc910074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57864/0 (socket says 172.21.15.26:57864) 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc914ddf640 1 -- 172.21.15.26:0/3591809363 learned_addr learned my addr 172.21.15.26:0/3591809363 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc90ffff640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc910107260 0x7fc910074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc9155e0640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc910074cd0 0x7fc9101a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc914ddf640 1 -- 172.21.15.26:0/3591809363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc910107260 msgr2=0x7fc910074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc914ddf640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc910107260 0x7fc910074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc914ddf640 1 -- 172.21.15.26:0/3591809363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc910074cd0 msgr2=0x7fc9101a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc914ddf640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc910074cd0 0x7fc9101a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc914ddf640 1 -- 172.21.15.26:0/3591809363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9101a6f40 con 0x7fc910102190 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc9155e0640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc910074cd0 0x7fc9101a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc914ddf640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc910102190 0x7fc910074250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc90400d950 tx=0x7fc90400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc90dffb640 1 -- 172.21.15.26:0/3591809363 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904017070 con 0x7fc910102190 2024-04-21T10:31:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc917043640 1 -- 172.21.15.26:0/3591809363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9101a7230 con 0x7fc910102190 2024-04-21T10:31:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc90dffb640 1 -- 172.21.15.26:0/3591809363 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc90400de80 con 0x7fc910102190 2024-04-21T10:31:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.366+0000 7fc90dffb640 1 -- 172.21.15.26:0/3591809363 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90400f620 con 0x7fc910102190 2024-04-21T10:31:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.367+0000 7fc917043640 1 -- 172.21.15.26:0/3591809363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9101a77d0 con 0x7fc910102190 2024-04-21T10:31:43.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc919c3a740 1 Processor -- start 2024-04-21T10:31:43.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc919c3a740 1 -- start start 2024-04-21T10:31:43.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc90dffb640 1 -- 172.21.15.26:0/3591809363 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fc904009c30 con 0x7fc910102190 2024-04-21T10:31:43.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc919c3a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 0x55f879607da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc919c3a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f879610980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc919c3a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f879610ec0 0x55f8796132b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc919c3a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f8795680b0 con 0x55f8796082e0 2024-04-21T10:31:43.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc919c3a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f879519680 con 0x55f879610ec0 2024-04-21T10:31:43.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc919c3a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f879525c60 con 0x55f8796079c0 2024-04-21T10:31:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc914ddf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f879610980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc9155e0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 0x55f879607da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc9155e0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 0x55f879607da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.26:47068/0 (socket says 172.21.15.26:47068) 2024-04-21T10:31:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc9155e0640 1 -- 172.21.15.26:0/1856590116 learned_addr learned my addr 172.21.15.26:0/1856590116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.370+0000 7fc90ffff640 1 --2- 172.21.15.26:0/1856590116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f879610ec0 0x55f8796132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.371+0000 7fc9155e0640 1 -- 172.21.15.26:0/1856590116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f879610ec0 msgr2=0x55f8796132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.371+0000 7fc9155e0640 1 --2- 172.21.15.26:0/1856590116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f879610ec0 0x55f8796132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.371+0000 7fc9155e0640 1 -- 172.21.15.26:0/1856590116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 msgr2=0x55f879610980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.371+0000 7fc9155e0640 1 --2- 172.21.15.26:0/1856590116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f879610980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.371+0000 7fc9155e0640 1 -- 172.21.15.26:0/1856590116 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87934ac90 con 0x55f8796079c0 2024-04-21T10:31:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.371+0000 7fc90ffff640 1 --2- 172.21.15.26:0/1856590116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f879610ec0 0x55f8796132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.371+0000 7fc90dffb640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc9000a3f50 0x7fc9000a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.371+0000 7fc9155e0640 1 --2- 172.21.15.26:0/1856590116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 0x55f879607da0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc908036000 tx=0x7fc908009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40ba6a8d9a6a7e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.371+0000 7fc8faffd640 1 -- 172.21.15.26:0/1856590116 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908043020 con 0x55f8796079c0 2024-04-21T10:31:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.371+0000 7fc914ddf640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc9000a3f50 0x7fc9000a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.371+0000 7fc8faffd640 1 -- 172.21.15.26:0/1856590116 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc908032970 con 0x55f8796079c0 2024-04-21T10:31:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.371+0000 7fc90dffb640 1 -- 172.21.15.26:0/3591809363 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fc9040d0660 con 0x7fc910102190 2024-04-21T10:31:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.372+0000 7fc919c3a740 1 -- 172.21.15.26:0/1856590116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 msgr2=0x55f879607da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.372+0000 7fc914ddf640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc9000a3f50 0x7fc9000a6410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc9040cdbe0 tx=0x7fc9040d6530 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.372+0000 7fc919c3a740 1 --2- 172.21.15.26:0/1856590116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 0x55f879607da0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc908036000 tx=0x7fc908009900 comp rx=0 tx=0).stop 2024-04-21T10:31:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.372+0000 7fc919c3a740 1 -- 172.21.15.26:0/1856590116 shutdown_connections 2024-04-21T10:31:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.372+0000 7fc919c3a740 1 --2- 172.21.15.26:0/1856590116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f879610ec0 0x55f8796132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.372+0000 7fc919c3a740 1 --2- 172.21.15.26:0/1856590116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f879610980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.372+0000 7fc919c3a740 1 --2- 172.21.15.26:0/1856590116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 0x55f879607da0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.372+0000 7fc919c3a740 1 -- 172.21.15.26:0/1856590116 >> 172.21.15.26:0/1856590116 conn(0x55f8796020f0 msgr2=0x55f8796024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.372+0000 7fc919c3a740 1 -- 172.21.15.26:0/1856590116 shutdown_connections 2024-04-21T10:31:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.372+0000 7fc919c3a740 1 -- 172.21.15.26:0/1856590116 wait complete. 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.373+0000 7fc919c3a740 1 -- start start 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.373+0000 7fc919c3a740 1 Processor -- start 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.373+0000 7fc919c3a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 0x55f87960de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.373+0000 7fc919c3a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f87960e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc90ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f87960e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc9155e0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 0x55f87960de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc90ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f87960e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57882/0 (socket says 172.21.15.26:57882) 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.373+0000 7fc919c3a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f879610ec0 0x55f87960e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc9155e0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 0x55f87960de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.26:47072/0 (socket says 172.21.15.26:47072) 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc90ffff640 1 -- 172.21.15.26:0/61608717 learned_addr learned my addr 172.21.15.26:0/61608717 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc919c3a740 1 -- 172.21.15.26:0/61608717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f87956b400 con 0x55f8796082e0 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc919c3a740 1 -- 172.21.15.26:0/61608717 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f879527840 con 0x55f879610ec0 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc919c3a740 1 -- 172.21.15.26:0/61608717 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55f879570720 con 0x55f8796079c0 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc9155e0640 1 -- 172.21.15.26:0/61608717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f879610ec0 msgr2=0x55f87960e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc9155e0640 1 --2- 172.21.15.26:0/61608717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f879610ec0 0x55f87960e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc9155e0640 1 -- 172.21.15.26:0/61608717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 msgr2=0x55f87960e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc9155e0640 1 --2- 172.21.15.26:0/61608717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f87960e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc9155e0640 1 -- 172.21.15.26:0/61608717 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8793ab420 con 0x55f8796079c0 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc90ffff640 1 --2- 172.21.15.26:0/61608717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f87960e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc919c3a740 10 client.0 _ll_register_callbacks cb 0x55f879578ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc919c3a740 1 -- 172.21.15.26:0/61608717 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc908032970 con 0x55f8796079c0 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc9155e0640 1 --2- 172.21.15.26:0/61608717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 0x55f87960de90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc908006010 tx=0x7fc90802c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feab1cab2d25603 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.374+0000 7fc8effff640 10 client.25273 ms_handle_connect on v2:172.21.15.190:3300/0 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.375+0000 7fc8effff640 1 -- 172.21.15.26:0/61608717 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908057050 con 0x55f8796079c0 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.375+0000 7fc8effff640 1 -- 172.21.15.26:0/61608717 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fc90802cda0 con 0x55f8796079c0 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.375+0000 7fc8effff640 1 -- 172.21.15.26:0/61608717 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc90804c030 con 0x55f8796079c0 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.375+0000 7fc919c3a740 10 client.25273 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.375+0000 7fc919c3a740 1 -- 172.21.15.26:0/61608717 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55f87923e580 con 0x55f8796079c0 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.375+0000 7fc8effff640 1 -- 172.21.15.26:0/61608717 <== mon.1 v2:172.21.15.190:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc90805a020 con 0x55f8796079c0 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.375+0000 7fc919c3a740 10 client.25273 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.375+0000 7fc919c3a740 10 client.25273 resolve_mds: validated gid 24496 aka daemon mds.i 2024-04-21T10:31:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.375+0000 7fc919c3a740 20 client.25273 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.375+0000 7fc919c3a740 1 --2- 172.21.15.26:0/61608717 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55f879620670 0x55f879622a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.375+0000 7fc919c3a740 4 client.25273 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:31:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.375+0000 7fc919c3a740 1 -- 172.21.15.26:0/61608717 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f879579b10 con 0x55f879620670 2024-04-21T10:31:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.376+0000 7fc90ffff640 1 --2- 172.21.15.26:0/61608717 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55f879620670 0x55f879622a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.376+0000 7fc90ffff640 1 --2- 172.21.15.26:0/61608717 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55f879620670 0x55f879622a60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c2f4dfb7ab7ce1 server_cookie=39c0411592e18f40 in_seq=0 out_seq=0 2024-04-21T10:31:43.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.376+0000 7fc8effff640 10 client.25273 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:43.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc8effff640 1 -- 172.21.15.26:0/61608717 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f879579b10 con 0x55f879620670 2024-04-21T10:31:43.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc8effff640 10 client.25273 handle_command_reply: tid=0 2024-04-21T10:31:43.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc919c3a740 1 client.25273 shutdown 2024-04-21T10:31:43.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc919c3a740 1 -- 172.21.15.26:0/61608717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 msgr2=0x55f87960de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc919c3a740 1 --2- 172.21.15.26:0/61608717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 0x55f87960de90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc908006010 tx=0x7fc90802c800 comp rx=0 tx=0).stop 2024-04-21T10:31:43.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc919c3a740 1 -- 172.21.15.26:0/61608717 shutdown_connections 2024-04-21T10:31:43.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc919c3a740 1 --2- 172.21.15.26:0/61608717 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55f879620670 0x55f879622a60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc8effff640 0 client.25273 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc919c3a740 1 --2- 172.21.15.26:0/61608717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f879610ec0 0x55f87960e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc919c3a740 1 --2- 172.21.15.26:0/61608717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f87960e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc919c3a740 1 --2- 172.21.15.26:0/61608717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796079c0 0x55f87960de90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc919c3a740 1 -- 172.21.15.26:0/61608717 >> 172.21.15.26:0/61608717 conn(0x55f8796020f0 msgr2=0x55f879603e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc919c3a740 1 -- 172.21.15.26:0/61608717 shutdown_connections 2024-04-21T10:31:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc919c3a740 1 -- 172.21.15.26:0/61608717 wait complete. 2024-04-21T10:31:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.377+0000 7fc919c3a740 20 client.25273 trim_cache size 0 max 16384 2024-04-21T10:31:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc919c3a740 1 Processor -- start 2024-04-21T10:31:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc919c3a740 1 -- start start 2024-04-21T10:31:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc919c3a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 0x55f8796105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc919c3a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f879600fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc919c3a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f879610ec0 0x55f87960c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc919c3a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101afe80 con 0x55f8796082e0 2024-04-21T10:31:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc90ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f879600fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc90ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f879600fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57888/0 (socket says 172.21.15.26:57888) 2024-04-21T10:31:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc90ffff640 1 -- 172.21.15.26:0/1986799674 learned_addr learned my addr 172.21.15.26:0/1986799674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc919c3a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101adcf0 con 0x55f8796079c0 2024-04-21T10:31:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc919c3a740 1 -- 172.21.15.26:0/1986799674 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101af7b0 con 0x55f879610ec0 2024-04-21T10:31:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc9155e0640 1 --2- 172.21.15.26:0/1986799674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 0x55f8796105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.413+0000 7fc9155e0640 1 --2- 172.21.15.26:0/1986799674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f879610ec0 0x55f87960c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.414+0000 7fc9155e0640 1 -- 172.21.15.26:0/1986799674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f879610ec0 msgr2=0x55f87960c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.414+0000 7fc9155e0640 1 --2- 172.21.15.26:0/1986799674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f879610ec0 0x55f87960c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.414+0000 7fc9155e0640 1 -- 172.21.15.26:0/1986799674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 msgr2=0x55f879600fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.414+0000 7fc9155e0640 1 --2- 172.21.15.26:0/1986799674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f879600fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.414+0000 7fc9155e0640 1 -- 172.21.15.26:0/1986799674 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f879608110 con 0x55f8796079c0 2024-04-21T10:31:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.414+0000 7fc90ffff640 1 --2- 172.21.15.26:0/1986799674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f879600fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:31:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.414+0000 7fc9155e0640 1 --2- 172.21.15.26:0/1986799674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 0x55f8796105c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc908032940 tx=0x7fc908041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3576bfe638872645 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.414+0000 7fc8fa7fc640 1 -- 172.21.15.26:0/1986799674 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908057030 con 0x55f8796079c0 2024-04-21T10:31:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.414+0000 7fc8fa7fc640 1 -- 172.21.15.26:0/1986799674 <== mon.2 v2:172.21.15.169:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc908032970 con 0x55f8796079c0 2024-04-21T10:31:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.414+0000 7fc8fa7fc640 1 -- 172.21.15.26:0/1986799674 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908043040 con 0x55f8796079c0 2024-04-21T10:31:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.415+0000 7fc919c3a740 1 -- 172.21.15.26:0/1986799674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 msgr2=0x55f8796105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.415+0000 7fc919c3a740 1 --2- 172.21.15.26:0/1986799674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 0x55f8796105c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc908032940 tx=0x7fc908041040 comp rx=0 tx=0).stop 2024-04-21T10:31:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.415+0000 7fc919c3a740 1 -- 172.21.15.26:0/1986799674 shutdown_connections 2024-04-21T10:31:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.415+0000 7fc919c3a740 1 --2- 172.21.15.26:0/1986799674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f879610ec0 0x55f87960c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.415+0000 7fc919c3a740 1 --2- 172.21.15.26:0/1986799674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8796082e0 0x55f879600fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.415+0000 7fc919c3a740 1 --2- 172.21.15.26:0/1986799674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 0x55f8796105c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.415+0000 7fc919c3a740 1 -- 172.21.15.26:0/1986799674 >> 172.21.15.26:0/1986799674 conn(0x55f8796020f0 msgr2=0x55f879603980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.415+0000 7fc919c3a740 1 -- 172.21.15.26:0/1986799674 shutdown_connections 2024-04-21T10:31:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.415+0000 7fc919c3a740 1 -- 172.21.15.26:0/1986799674 wait complete. 2024-04-21T10:31:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.416+0000 7fc919c3a740 1 -- start start 2024-04-21T10:31:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.416+0000 7fc919c3a740 1 Processor -- start 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.416+0000 7fc919c3a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 0x55f879619280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.416+0000 7fc919c3a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796082e0 0x55f879614690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.416+0000 7fc919c3a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f879610ec0 0x55f879614c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.416+0000 7fc919c3a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101aea10 con 0x55f879610ec0 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.416+0000 7fc919c3a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101ae340 con 0x55f8796079c0 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.416+0000 7fc919c3a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9101af0e0 con 0x55f8796082e0 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.416+0000 7fc919c3a740 10 client.0 _ll_register_callbacks cb 0x7fc9040d6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.417+0000 7fc9155e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f879610ec0 0x55f879614c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.417+0000 7fc9155e0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 0x55f879619280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.417+0000 7fc9155e0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 0x55f879619280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.26:50768/0 (socket says 172.21.15.26:50768) 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.417+0000 7fc9155e0640 1 -- 172.21.15.26:0/2644158982 learned_addr learned my addr 172.21.15.26:0/2644158982 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.417+0000 7fc90ffff640 1 --2- 172.21.15.26:0/2644158982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796082e0 0x55f879614690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.417+0000 7fc9155e0640 1 -- 172.21.15.26:0/2644158982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796082e0 msgr2=0x55f879614690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.417+0000 7fc9155e0640 1 --2- 172.21.15.26:0/2644158982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796082e0 0x55f879614690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.417+0000 7fc9155e0640 1 -- 172.21.15.26:0/2644158982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f879610ec0 msgr2=0x55f879614c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.417+0000 7fc9155e0640 1 --2- 172.21.15.26:0/2644158982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f879610ec0 0x55f879614c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.417+0000 7fc9155e0640 1 -- 172.21.15.26:0/2644158982 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f879615460 con 0x55f8796079c0 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.417+0000 7fc9155e0640 1 --2- 172.21.15.26:0/2644158982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 0x55f879619280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc908003d10 tx=0x7fc9080095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d5f9633677e252b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.417+0000 7fc8effff640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc8effff640 1 -- 172.21.15.26:0/2644158982 <== mon.2 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908057040 con 0x55f8796079c0 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc919c3a740 1 -- 172.21.15.26:0/2644158982 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc908032970 con 0x55f8796079c0 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc8effff640 1 -- 172.21.15.26:0/2644158982 <== mon.2 v2:172.21.15.169:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fc908003970 con 0x55f8796079c0 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc8effff640 1 -- 172.21.15.26:0/2644158982 <== mon.2 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc90803f470 con 0x55f8796079c0 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc919c3a740 10 client.25190 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc919c3a740 1 -- 172.21.15.26:0/2644158982 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f879686b60 con 0x55f8796079c0 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc8effff640 1 -- 172.21.15.26:0/2644158982 <== mon.2 v2:172.21.15.169:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc908032970 con 0x55f8796079c0 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc8effff640 1 -- 172.21.15.26:0/2644158982 <== mon.2 v2:172.21.15.169:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc90805a040 con 0x55f8796079c0 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc919c3a740 10 client.25190 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc919c3a740 10 client.25190 resolve_mds: validated gid 24496 aka daemon mds.i 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc919c3a740 20 client.25190 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc919c3a740 1 --2- 172.21.15.26:0/2644158982 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55f879620670 0x55f879689290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc919c3a740 4 client.25190 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 78.95, "await": true}] 2024-04-21T10:31:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.418+0000 7fc919c3a740 1 -- 172.21.15.26:0/2644158982 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 78.95, "await": true}) -- 0x55f879573980 con 0x55f879620670 2024-04-21T10:31:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.419+0000 7fc90ffff640 1 --2- 172.21.15.26:0/2644158982 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55f879620670 0x55f879689290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.419+0000 7fc90ffff640 1 --2- 172.21.15.26:0/2644158982 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55f879620670 0x55f879689290 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4852190eece1aa server_cookie=26ad9bce04eaa1e in_seq=0 out_seq=0 2024-04-21T10:31:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.419+0000 7fc8effff640 10 client.25190 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc8effff640 1 -- 172.21.15.26:0/2644158982 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55f879573980 con 0x55f879620670 2024-04-21T10:31:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc8effff640 10 client.25190 handle_command_reply: tid=0 2024-04-21T10:31:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc919c3a740 1 client.25190 shutdown 2024-04-21T10:31:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc919c3a740 1 -- 172.21.15.26:0/2644158982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 msgr2=0x55f879619280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc919c3a740 1 --2- 172.21.15.26:0/2644158982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 0x55f879619280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc908003d10 tx=0x7fc9080095e0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc919c3a740 1 -- 172.21.15.26:0/2644158982 shutdown_connections 2024-04-21T10:31:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc919c3a740 1 --2- 172.21.15.26:0/2644158982 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55f879620670 0x55f879689290 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc919c3a740 1 --2- 172.21.15.26:0/2644158982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f879610ec0 0x55f879614c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc919c3a740 1 --2- 172.21.15.26:0/2644158982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f8796082e0 0x55f879614690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc919c3a740 1 --2- 172.21.15.26:0/2644158982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8796079c0 0x55f879619280 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc919c3a740 1 -- 172.21.15.26:0/2644158982 >> 172.21.15.26:0/2644158982 conn(0x55f8796020f0 msgr2=0x55f87960f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc8effff640 0 client.25190 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.427+0000 7fc919c3a740 1 -- 172.21.15.26:0/2644158982 shutdown_connections 2024-04-21T10:31:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.428+0000 7fc919c3a740 1 -- 172.21.15.26:0/2644158982 wait complete. 2024-04-21T10:31:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.428+0000 7fc919c3a740 20 client.25190 trim_cache size 0 max 16384 2024-04-21T10:31:43.428 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:31:43.428 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 21, 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24496, 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 6, 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "dd4a255d": { 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 6, 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 30.0, 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 79.0, 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-04-21T10:31:43.429 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-04-21T10:31:43.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "QUIESCED", 2024-04-21T10:31:43.430 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-04-21T10:31:43.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:43.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:43.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:43.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:43.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:31:43.430 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:31:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.428+0000 7fc917043640 1 -- 172.21.15.26:0/3591809363 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc9000a3f50 msgr2=0x7fc9000a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.428+0000 7fc917043640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc9000a3f50 0x7fc9000a6410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc9040cdbe0 tx=0x7fc9040d6530 comp rx=0 tx=0).stop 2024-04-21T10:31:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.429+0000 7fc917043640 1 -- 172.21.15.26:0/3591809363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc910102190 msgr2=0x7fc910074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.429+0000 7fc917043640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc910102190 0x7fc910074250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc90400d950 tx=0x7fc90400dd50 comp rx=0 tx=0).stop 2024-04-21T10:31:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.429+0000 7fc917043640 1 -- 172.21.15.26:0/3591809363 shutdown_connections 2024-04-21T10:31:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.429+0000 7fc917043640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fc9000a3f50 0x7fc9000a6410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.429+0000 7fc917043640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc910074cd0 0x7fc9101a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.429+0000 7fc917043640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc910107260 0x7fc910074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.429+0000 7fc917043640 1 --2- 172.21.15.26:0/3591809363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc910102190 0x7fc910074250 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.429+0000 7fc917043640 1 -- 172.21.15.26:0/3591809363 >> 172.21.15.26:0/3591809363 conn(0x7fc910079310 msgr2=0x7fc9100ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.429+0000 7fc917043640 1 -- 172.21.15.26:0/3591809363 shutdown_connections 2024-04-21T10:31:43.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:31:43.429+0000 7fc917043640 1 -- 172.21.15.26:0/3591809363 wait complete. 2024-04-21T10:31:43.443 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: dd4a255d, quiesce duration: 18.95 2024-04-21T10:31:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:43 smithi026 ceph-mon[27965]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T10:31:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:43 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1763723483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:43 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/4267482641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:43.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:43 smithi169 ceph-mon[31469]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T10:31:43.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:43 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1763723483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:43.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:43 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/4267482641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:43.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:43 smithi190 ceph-mon[31423]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T10:31:43.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:43 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1763723483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:43.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:43 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/4267482641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:31:45.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:45 smithi026 ceph-mon[27965]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T10:31:45.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:45 smithi169 ceph-mon[31469]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T10:31:45.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:45 smithi190 ceph-mon[31423]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T10:31:47.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:47 smithi026 ceph-mon[27965]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-21T10:31:47.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:47 smithi169 ceph-mon[31469]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-21T10:31:47.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:47 smithi190 ceph-mon[31423]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-21T10:31:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:49 smithi026 ceph-mon[27965]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-21T10:31:49.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:49 smithi169 ceph-mon[31469]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-21T10:31:49.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:49 smithi190 ceph-mon[31423]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-21T10:31:51.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:51 smithi026 ceph-mon[27965]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-21T10:31:51.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:51 smithi169 ceph-mon[31469]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-21T10:31:51.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:51 smithi190 ceph-mon[31423]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-21T10:31:53.205 INFO:tasks.workunit.client.0.smithi026.stderr:Note: switching to 'b8ddeda347de6c45510a6df67440a1d298c37030'. 2024-04-21T10:31:53.205 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:31:53.205 INFO:tasks.workunit.client.0.smithi026.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-21T10:31:53.205 INFO:tasks.workunit.client.0.smithi026.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-21T10:31:53.205 INFO:tasks.workunit.client.0.smithi026.stderr:state without impacting any branches by switching back to a branch. 2024-04-21T10:31:53.205 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:31:53.205 INFO:tasks.workunit.client.0.smithi026.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-21T10:31:53.205 INFO:tasks.workunit.client.0.smithi026.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-21T10:31:53.205 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:31:53.205 INFO:tasks.workunit.client.0.smithi026.stderr: git switch -c 2024-04-21T10:31:53.205 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:31:53.205 INFO:tasks.workunit.client.0.smithi026.stderr:Or undo this operation with: 2024-04-21T10:31:53.206 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:31:53.206 INFO:tasks.workunit.client.0.smithi026.stderr: git switch - 2024-04-21T10:31:53.206 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:31:53.206 INFO:tasks.workunit.client.0.smithi026.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-21T10:31:53.206 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:31:53.206 INFO:tasks.workunit.client.0.smithi026.stderr:HEAD is now at b8ddeda347d qa/suites/fs: add quiescer to the fs suite 2024-04-21T10:31:53.207 DEBUG:teuthology.orchestra.run.smithi026:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-21T10:31:53.267 INFO:tasks.workunit.client.0.smithi026.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-21T10:31:53.270 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-21T10:31:53.270 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-21T10:31:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.293+0000 7facb9025640 1 -- 172.21.15.26:0/942626320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb4103af0 msgr2=0x7facb410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.293+0000 7facb9025640 1 --2- 172.21.15.26:0/942626320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb4103af0 0x7facb410bfd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009900 tx=0x7fac9c02bcd0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.293+0000 7facb9025640 1 -- 172.21.15.26:0/942626320 shutdown_connections 2024-04-21T10:31:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.293+0000 7facb9025640 1 --2- 172.21.15.26:0/942626320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facb410c5a0 0x7facb410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.293+0000 7facb9025640 1 --2- 172.21.15.26:0/942626320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb4103af0 0x7facb410bfd0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.293+0000 7facb9025640 1 --2- 172.21.15.26:0/942626320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facb41031d0 0x7facb41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.293+0000 7facb9025640 1 -- 172.21.15.26:0/942626320 >> 172.21.15.26:0/942626320 conn(0x7facb4077fa0 msgr2=0x7facb40783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.293+0000 7facb9025640 1 -- 172.21.15.26:0/942626320 shutdown_connections 2024-04-21T10:31:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.293+0000 7facb9025640 1 -- 172.21.15.26:0/942626320 wait complete. 2024-04-21T10:31:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.294+0000 7facb9025640 1 Processor -- start 2024-04-21T10:31:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.294+0000 7facb9025640 1 -- start start 2024-04-21T10:31:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.294+0000 7facb9025640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb41031d0 0x7facb41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.294+0000 7facb9025640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facb4103af0 0x7facb4198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.294+0000 7facb9025640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facb410c5a0 0x7facb419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.294+0000 7facb9025640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facb4112230 con 0x7facb41031d0 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.294+0000 7facb9025640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7facb41120b0 con 0x7facb4103af0 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.294+0000 7facb9025640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7facb41123b0 con 0x7facb410c5a0 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.294+0000 7facb2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb41031d0 0x7facb41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.294+0000 7facb2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb41031d0 0x7facb41987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39010/0 (socket says 172.21.15.26:39010) 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.294+0000 7facb2d9d640 1 -- 172.21.15.26:0/1479677939 learned_addr learned my addr 172.21.15.26:0/1479677939 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.294+0000 7facb259c640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facb4103af0 0x7facb4198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.295+0000 7facb359e640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facb410c5a0 0x7facb419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.295+0000 7facb2d9d640 1 -- 172.21.15.26:0/1479677939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facb410c5a0 msgr2=0x7facb419d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.295+0000 7facb2d9d640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facb410c5a0 0x7facb419d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.295+0000 7facb2d9d640 1 -- 172.21.15.26:0/1479677939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facb4103af0 msgr2=0x7facb4198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.295+0000 7facb2d9d640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facb4103af0 0x7facb4198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.295+0000 7facb2d9d640 1 -- 172.21.15.26:0/1479677939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb419d750 con 0x7facb41031d0 2024-04-21T10:31:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.295+0000 7facb2d9d640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb41031d0 0x7facb41987b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faca400ba50 tx=0x7faca400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.295+0000 7facabfff640 1 -- 172.21.15.26:0/1479677939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4019070 con 0x7facb41031d0 2024-04-21T10:31:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.295+0000 7facabfff640 1 -- 172.21.15.26:0/1479677939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca400cbe0 con 0x7facb41031d0 2024-04-21T10:31:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.295+0000 7facb9025640 1 -- 172.21.15.26:0/1479677939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb419da40 con 0x7facb41031d0 2024-04-21T10:31:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.295+0000 7facabfff640 1 -- 172.21.15.26:0/1479677939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca40075d0 con 0x7facb41031d0 2024-04-21T10:31:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.295+0000 7facb9025640 1 -- 172.21.15.26:0/1479677939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb41a52d0 con 0x7facb41031d0 2024-04-21T10:31:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.297+0000 7facabfff640 1 -- 172.21.15.26:0/1479677939 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7faca401c070 con 0x7facb41031d0 2024-04-21T10:31:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.298+0000 7facbbc12740 1 Processor -- start 2024-04-21T10:31:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.298+0000 7facbbc12740 1 -- start start 2024-04-21T10:31:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.298+0000 7facbbc12740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2862098d0 0x55d286208620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.298+0000 7facbbc12740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d286208b60 0x55d286208fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.298+0000 7facbbc12740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 0x55d2862144d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.298+0000 7facbbc12740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d28616d8d0 con 0x55d2862120e0 2024-04-21T10:31:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.298+0000 7facbbc12740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d28610ae80 con 0x55d2862098d0 2024-04-21T10:31:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.298+0000 7facbbc12740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d286169890 con 0x55d286208b60 2024-04-21T10:31:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.298+0000 7facabfff640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7faca00a3f70 0x7faca00a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.298+0000 7facb259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 0x55d2862144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.298+0000 7facb259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 0x55d2862144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39026/0 (socket says 172.21.15.26:39026) 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.298+0000 7facb259c640 1 -- 172.21.15.26:0/2341111124 learned_addr learned my addr 172.21.15.26:0/2341111124 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7facb359e640 1 --2- 172.21.15.26:0/2341111124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2862098d0 0x55d286208620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7facb2d9d640 1 --2- 172.21.15.26:0/2341111124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d286208b60 0x55d286208fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7facb359e640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7faca00a3f70 0x7faca00a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7facabfff640 1 -- 172.21.15.26:0/1479677939 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7faca40cff90 con 0x7facb41031d0 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7facb259c640 1 -- 172.21.15.26:0/2341111124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d286208b60 msgr2=0x55d286208fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7facb259c640 1 --2- 172.21.15.26:0/2341111124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d286208b60 0x55d286208fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7facb259c640 1 -- 172.21.15.26:0/2341111124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2862098d0 msgr2=0x55d286208620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7facb259c640 1 --2- 172.21.15.26:0/2341111124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2862098d0 0x55d286208620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7facb259c640 1 -- 172.21.15.26:0/2341111124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d28612c320 con 0x55d2862120e0 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7facb259c640 1 --2- 172.21.15.26:0/2341111124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 0x55d2862144d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009900 tx=0x7fac9c02c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b6eaa140f183a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7facb359e640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7faca00a3f70 0x7faca00a6430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7facac009fd0 tx=0x7facac005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7faca97fa640 1 -- 172.21.15.26:0/2341111124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c004180 con 0x55d2862120e0 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7faca97fa640 1 -- 172.21.15.26:0/2341111124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac9c0054f0 con 0x55d2862120e0 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.299+0000 7faca97fa640 1 -- 172.21.15.26:0/2341111124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0046c0 con 0x55d2862120e0 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.300+0000 7facbbc12740 1 -- 172.21.15.26:0/2341111124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 msgr2=0x55d2862144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.300+0000 7facbbc12740 1 --2- 172.21.15.26:0/2341111124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 0x55d2862144d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009900 tx=0x7fac9c02c760 comp rx=0 tx=0).stop 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.300+0000 7facbbc12740 1 -- 172.21.15.26:0/2341111124 shutdown_connections 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.300+0000 7facbbc12740 1 --2- 172.21.15.26:0/2341111124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 0x55d2862144d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.300+0000 7facbbc12740 1 --2- 172.21.15.26:0/2341111124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d286208b60 0x55d286208fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.300+0000 7facbbc12740 1 --2- 172.21.15.26:0/2341111124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2862098d0 0x55d286208620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.300+0000 7facbbc12740 1 -- 172.21.15.26:0/2341111124 >> 172.21.15.26:0/2341111124 conn(0x55d2861cfc90 msgr2=0x55d286203850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.300+0000 7facbbc12740 1 -- 172.21.15.26:0/2341111124 shutdown_connections 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.300+0000 7facbbc12740 1 -- 172.21.15.26:0/2341111124 wait complete. 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.300+0000 7facbbc12740 1 -- start start 2024-04-21T10:31:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.300+0000 7facbbc12740 1 Processor -- start 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facbbc12740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d286208b60 0x55d2862119f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facbbc12740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862098d0 0x55d28620ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facbbc12740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862120e0 0x55d28620f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facbbc12740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d286214db0 con 0x55d2862098d0 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facbbc12740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d2860f5eb0 con 0x55d286208b60 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facbbc12740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55d286214f30 con 0x55d2862120e0 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facbbc12740 10 client.0 _ll_register_callbacks cb 0x55d286122b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facb259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862098d0 0x55d28620ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facb259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862098d0 0x55d28620ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39036/0 (socket says 172.21.15.26:39036) 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facb259c640 1 -- 172.21.15.26:0/3459090329 learned_addr learned my addr 172.21.15.26:0/3459090329 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facb359e640 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862120e0 0x55d28620f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facb259c640 1 -- 172.21.15.26:0/3459090329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862120e0 msgr2=0x55d28620f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facb2d9d640 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d286208b60 0x55d2862119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facb259c640 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862120e0 0x55d28620f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facb259c640 1 -- 172.21.15.26:0/3459090329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d286208b60 msgr2=0x55d2862119f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facb259c640 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d286208b60 0x55d2862119f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facb259c640 1 -- 172.21.15.26:0/3459090329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d285e80210 con 0x55d2862098d0 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facb259c640 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862098d0 0x55d28620ed80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fac9c02c210 tx=0x7fac9c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcbdc666eb68958 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7fac9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7fac9a7fc640 1 -- 172.21.15.26:0/3459090329 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c003a00 con 0x55d2862098d0 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7facbbc12740 1 -- 172.21.15.26:0/3459090329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac9c0054f0 con 0x55d2862098d0 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.301+0000 7fac9a7fc640 1 -- 172.21.15.26:0/3459090329 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fac9c003ba0 con 0x55d2862098d0 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.302+0000 7fac9a7fc640 1 -- 172.21.15.26:0/3459090329 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fac9c05a050 con 0x55d2862098d0 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.302+0000 7facbbc12740 10 client.15702 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.302+0000 7facbbc12740 1 -- 172.21.15.26:0/3459090329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d285f4c620 con 0x55d2862098d0 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.302+0000 7fac9a7fc640 1 -- 172.21.15.26:0/3459090329 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac9c0054f0 con 0x55d2862098d0 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.302+0000 7fac9a7fc640 1 -- 172.21.15.26:0/3459090329 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fac9c03f430 con 0x55d2862098d0 2024-04-21T10:31:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.302+0000 7facbbc12740 10 client.15702 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.302+0000 7facbbc12740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.302+0000 7facbbc12740 20 client.15702 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.302+0000 7facbbc12740 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55d286221ec0 0x55d2862242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.302+0000 7facbbc12740 4 client.15702 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.302+0000 7facbbc12740 1 -- 172.21.15.26:0/3459090329 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d286181880 con 0x55d286221ec0 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.302+0000 7facb2d9d640 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55d286221ec0 0x55d2862242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.303+0000 7fac9a7fc640 10 client.15702 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.303+0000 7facb2d9d640 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55d286221ec0 0x55d2862242b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857dd6047ae67f46 server_cookie=5ecc2886dfc233dc in_seq=0 out_seq=0 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.303+0000 7fac9a7fc640 1 -- 172.21.15.26:0/3459090329 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d286181880 con 0x55d286221ec0 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.303+0000 7fac9a7fc640 10 client.15702 handle_command_reply: tid=0 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7facbbc12740 1 client.15702 shutdown 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7facbbc12740 1 -- 172.21.15.26:0/3459090329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862098d0 msgr2=0x55d28620ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7facbbc12740 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862098d0 0x55d28620ed80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fac9c02c210 tx=0x7fac9c02bcd0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7facbbc12740 1 -- 172.21.15.26:0/3459090329 shutdown_connections 2024-04-21T10:31:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7facbbc12740 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55d286221ec0 0x55d2862242b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7facbbc12740 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862120e0 0x55d28620f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7facbbc12740 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862098d0 0x55d28620ed80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7facbbc12740 1 --2- 172.21.15.26:0/3459090329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d286208b60 0x55d2862119f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7fac9a7fc640 0 client.15702 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7facbbc12740 1 -- 172.21.15.26:0/3459090329 >> 172.21.15.26:0/3459090329 conn(0x55d2861cfc90 msgr2=0x55d2862055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7facbbc12740 1 -- 172.21.15.26:0/3459090329 shutdown_connections 2024-04-21T10:31:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7facbbc12740 1 -- 172.21.15.26:0/3459090329 wait complete. 2024-04-21T10:31:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.304+0000 7facbbc12740 20 client.15702 trim_cache size 0 max 16384 2024-04-21T10:31:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facbbc12740 1 Processor -- start 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facbbc12740 1 -- start start 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facbbc12740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d286208b60 0x55d28620a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facbbc12740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862098d0 0x55d28621dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facbbc12740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2862120e0 0x55d28620f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facbbc12740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facb41b0350 con 0x55d286208b60 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facbbc12740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7facb41adc20 con 0x55d2862120e0 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facbbc12740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7facb41aba70 con 0x55d2862098d0 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facb2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d286208b60 0x55d28620a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facb2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d286208b60 0x55d28620a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39050/0 (socket says 172.21.15.26:39050) 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facb2d9d640 1 -- 172.21.15.26:0/1075730202 learned_addr learned my addr 172.21.15.26:0/1075730202 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facb259c640 1 --2- 172.21.15.26:0/1075730202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862098d0 0x55d28621dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facb359e640 1 --2- 172.21.15.26:0/1075730202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2862120e0 0x55d28620f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facb2d9d640 1 -- 172.21.15.26:0/1075730202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862098d0 msgr2=0x55d28621dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facb2d9d640 1 --2- 172.21.15.26:0/1075730202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862098d0 0x55d28621dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facb2d9d640 1 -- 172.21.15.26:0/1075730202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2862120e0 msgr2=0x55d28620f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facb2d9d640 1 --2- 172.21.15.26:0/1075730202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2862120e0 0x55d28620f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facb2d9d640 1 -- 172.21.15.26:0/1075730202 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d286207e30 con 0x55d286208b60 2024-04-21T10:31:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.319+0000 7facb359e640 1 --2- 172.21.15.26:0/1075730202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2862120e0 0x55d28620f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:31:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.320+0000 7facb2d9d640 1 --2- 172.21.15.26:0/1075730202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d286208b60 0x55d28620a630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faca4098320 tx=0x7faca4022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fcb31d56b6a4d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.320+0000 7faca8ff9640 1 -- 172.21.15.26:0/1075730202 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca400cd90 con 0x55d286208b60 2024-04-21T10:31:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.320+0000 7faca8ff9640 1 -- 172.21.15.26:0/1075730202 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca409a1c0 con 0x55d286208b60 2024-04-21T10:31:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.320+0000 7faca8ff9640 1 -- 172.21.15.26:0/1075730202 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca409c120 con 0x55d286208b60 2024-04-21T10:31:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.320+0000 7facbbc12740 1 -- 172.21.15.26:0/1075730202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d286208b60 msgr2=0x55d28620a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.320+0000 7facbbc12740 1 --2- 172.21.15.26:0/1075730202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d286208b60 0x55d28620a630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faca4098320 tx=0x7faca4022f90 comp rx=0 tx=0).stop 2024-04-21T10:31:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.320+0000 7facbbc12740 1 -- 172.21.15.26:0/1075730202 shutdown_connections 2024-04-21T10:31:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.320+0000 7facbbc12740 1 --2- 172.21.15.26:0/1075730202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2862120e0 0x55d28620f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.320+0000 7facbbc12740 1 --2- 172.21.15.26:0/1075730202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862098d0 0x55d28621dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.320+0000 7facbbc12740 1 --2- 172.21.15.26:0/1075730202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d286208b60 0x55d28620a630 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.320+0000 7facbbc12740 1 -- 172.21.15.26:0/1075730202 >> 172.21.15.26:0/1075730202 conn(0x55d2861cfc90 msgr2=0x55d28620d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facbbc12740 1 -- 172.21.15.26:0/1075730202 shutdown_connections 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facbbc12740 1 -- 172.21.15.26:0/1075730202 wait complete. 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facbbc12740 1 -- start start 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facbbc12740 1 Processor -- start 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facbbc12740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d286208b60 0x55d286210e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facbbc12740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862098d0 0x55d2862113a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facbbc12740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 0x55d2862118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facbbc12740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facb41ac0e0 con 0x55d2862120e0 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facbbc12740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7facb41ad550 con 0x55d286208b60 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facbbc12740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7facb41ac7b0 con 0x55d2862098d0 2024-04-21T10:31:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facbbc12740 10 client.0 _ll_register_callbacks cb 0x7facac009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facb359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 0x55d2862118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facb359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 0x55d2862118e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39054/0 (socket says 172.21.15.26:39054) 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facb359e640 1 -- 172.21.15.26:0/832380803 learned_addr learned my addr 172.21.15.26:0/832380803 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facb2d9d640 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d286208b60 0x55d286210e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.321+0000 7facb259c640 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862098d0 0x55d2862113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7facb359e640 1 -- 172.21.15.26:0/832380803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862098d0 msgr2=0x55d2862113a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7facb359e640 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862098d0 0x55d2862113a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7facb359e640 1 -- 172.21.15.26:0/832380803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d286208b60 msgr2=0x55d286210e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7facb359e640 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d286208b60 0x55d286210e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7facb359e640 1 -- 172.21.15.26:0/832380803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d28628b240 con 0x55d2862120e0 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7facb359e640 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 0x55d2862118e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7facac009410 tx=0x7facac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60388bcf3fe501c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7fac9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7fac9a7fc640 1 -- 172.21.15.26:0/832380803 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac00dc70 con 0x55d2862120e0 2024-04-21T10:31:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7fac9a7fc640 1 -- 172.21.15.26:0/832380803 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7facac01b070 con 0x55d2862120e0 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7facbbc12740 1 -- 172.21.15.26:0/832380803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faca409a1c0 con 0x55d2862120e0 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7fac9a7fc640 1 -- 172.21.15.26:0/832380803 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7facac00de10 con 0x55d2862120e0 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7facbbc12740 10 client.15708 fetch_fsmap learned FSMap version 21 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7facbbc12740 1 -- 172.21.15.26:0/832380803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d28628c4c0 con 0x55d2862120e0 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7fac9a7fc640 1 -- 172.21.15.26:0/832380803 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca409a1c0 con 0x55d2862120e0 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.322+0000 7fac9a7fc640 1 -- 172.21.15.26:0/832380803 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7facac033880 con 0x55d2862120e0 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.323+0000 7facbbc12740 10 client.15708 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.323+0000 7facbbc12740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.323+0000 7facbbc12740 20 client.15708 populate_metadata read hostname 'smithi026' 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.323+0000 7facbbc12740 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55d28628cc50 0x55d28628f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.323+0000 7facbbc12740 4 client.15708 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.323+0000 7facbbc12740 1 -- 172.21.15.26:0/832380803 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d28610d3d0 con 0x55d28628cc50 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.323+0000 7facb2d9d640 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55d28628cc50 0x55d28628f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.323+0000 7facb2d9d640 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55d28628cc50 0x55d28628f070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=922863d2eac43575 server_cookie=ffe0e13a8a4624a9 in_seq=0 out_seq=0 2024-04-21T10:31:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.323+0000 7fac9a7fc640 10 client.15708 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7fac9a7fc640 1 -- 172.21.15.26:0/832380803 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d28610d3d0 con 0x55d28628cc50 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7fac9a7fc640 10 client.15708 handle_command_reply: tid=0 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7facbbc12740 1 client.15708 shutdown 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7facbbc12740 1 -- 172.21.15.26:0/832380803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 msgr2=0x55d2862118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7facbbc12740 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 0x55d2862118e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7facac009410 tx=0x7facac0046c0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7facbbc12740 1 -- 172.21.15.26:0/832380803 shutdown_connections 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7facbbc12740 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55d28628cc50 0x55d28628f070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7facbbc12740 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2862120e0 0x55d2862118e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7facbbc12740 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d2862098d0 0x55d2862113a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7fac9a7fc640 0 client.15708 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7facbbc12740 1 --2- 172.21.15.26:0/832380803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d286208b60 0x55d286210e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7facbbc12740 1 -- 172.21.15.26:0/832380803 >> 172.21.15.26:0/832380803 conn(0x55d2861cfc90 msgr2=0x55d286204c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7facbbc12740 1 -- 172.21.15.26:0/832380803 shutdown_connections 2024-04-21T10:31:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7facbbc12740 1 -- 172.21.15.26:0/832380803 wait complete. 2024-04-21T10:31:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.327+0000 7facbbc12740 20 client.15708 trim_cache size 0 max 16384 2024-04-21T10:31:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-04-21T10:31:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-04-21T10:31:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-04-21T10:31:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-04-21T10:31:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.328+0000 7facb9025640 1 -- 172.21.15.26:0/1479677939 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7faca00a3f70 msgr2=0x7faca00a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.328+0000 7facb9025640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7faca00a3f70 0x7faca00a6430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7facac009fd0 tx=0x7facac005f90 comp rx=0 tx=0).stop 2024-04-21T10:31:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.328+0000 7facb9025640 1 -- 172.21.15.26:0/1479677939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb41031d0 msgr2=0x7facb41987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:31:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.328+0000 7facb9025640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb41031d0 0x7facb41987b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faca400ba50 tx=0x7faca400bf10 comp rx=0 tx=0).stop 2024-04-21T10:31:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.328+0000 7facb9025640 1 -- 172.21.15.26:0/1479677939 shutdown_connections 2024-04-21T10:31:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.328+0000 7facb9025640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7faca00a3f70 0x7faca00a6430 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.328+0000 7facb9025640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facb410c5a0 0x7facb419d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.328+0000 7facb9025640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facb4103af0 0x7facb4198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.328+0000 7facb9025640 1 --2- 172.21.15.26:0/1479677939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facb41031d0 0x7facb41987b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:31:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.328+0000 7facb9025640 1 -- 172.21.15.26:0/1479677939 >> 172.21.15.26:0/1479677939 conn(0x7facb4077fa0 msgr2=0x7facb40ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:31:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.328+0000 7facb9025640 1 -- 172.21.15.26:0/1479677939 shutdown_connections 2024-04-21T10:31:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:31:53.328+0000 7facb9025640 1 -- 172.21.15.26:0/1479677939 wait complete. 2024-04-21T10:31:53.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:53 smithi026 ceph-mon[27965]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:31:53.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:53 smithi169 ceph-mon[31469]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:31:53.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:53 smithi190 ceph-mon[31423]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:31:53.892 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-21T10:31:53.947 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-21T10:31:53.988 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-21T10:31:53.989 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-21T10:31:53.989 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-21T10:31:54.030 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-21T10:31:54.034 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:31:54.034 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-21T10:31:54.050 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-21T10:31:54.051 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-21T10:31:54.051 DEBUG:teuthology.orchestra.run.smithi026:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b8ddeda347de6c45510a6df67440a1d298c37030 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-21T10:31:55.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:55 smithi026 ceph-mon[27965]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-21T10:31:55.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:55 smithi169 ceph-mon[31469]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-21T10:31:55.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:55 smithi190 ceph-mon[31423]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-21T10:31:57.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:57 smithi026 ceph-mon[27965]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-21T10:31:57.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:57 smithi169 ceph-mon[31469]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-21T10:31:57.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:57 smithi190 ceph-mon[31423]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-21T10:31:59.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:31:59 smithi026 ceph-mon[27965]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:31:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:31:59 smithi169 ceph-mon[31469]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:31:59.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:31:59 smithi190 ceph-mon[31423]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-21T10:32:01.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:01 smithi026 ceph-mon[27965]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-21T10:32:01.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:01 smithi169 ceph-mon[31469]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-21T10:32:01.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:01 smithi190 ceph-mon[31423]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-21T10:32:02.395 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:32:02.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.510+0000 7ff67b631640 1 -- 172.21.15.26:0/2467549869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6741011a0 msgr2=0x7ff674101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.510+0000 7ff67b631640 1 --2- 172.21.15.26:0/2467549869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6741011a0 0x7ff674101620 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff664009930 tx=0x7ff66402bc30 comp rx=0 tx=0).stop 2024-04-21T10:32:02.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.510+0000 7ff67b631640 1 -- 172.21.15.26:0/2467549869 shutdown_connections 2024-04-21T10:32:02.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.510+0000 7ff67b631640 1 --2- 172.21.15.26:0/2467549869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff674101b60 0x7ff67410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.510+0000 7ff67b631640 1 --2- 172.21.15.26:0/2467549869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6741011a0 0x7ff674101620 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.510+0000 7ff67b631640 1 --2- 172.21.15.26:0/2467549869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff674105e30 0x7ff674106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.510+0000 7ff67b631640 1 -- 172.21.15.26:0/2467549869 >> 172.21.15.26:0/2467549869 conn(0x7ff674077e30 msgr2=0x7ff674078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.510+0000 7ff67b631640 1 -- 172.21.15.26:0/2467549869 shutdown_connections 2024-04-21T10:32:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.510+0000 7ff67b631640 1 -- 172.21.15.26:0/2467549869 wait complete. 2024-04-21T10:32:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff67b631640 1 Processor -- start 2024-04-21T10:32:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff67b631640 1 -- start start 2024-04-21T10:32:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff67b631640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6741011a0 0x7ff674197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff67b631640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff674101b60 0x7ff674197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff67b631640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff674105e30 0x7ff67419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff67b631640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741110e0 con 0x7ff674101b60 2024-04-21T10:32:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff67b631640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff674110f60 con 0x7ff6741011a0 2024-04-21T10:32:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff67b631640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff674111260 con 0x7ff674105e30 2024-04-21T10:32:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff678bcc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff674101b60 0x7ff674197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff678bcc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff674101b60 0x7ff674197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33704/0 (socket says 172.21.15.26:33704) 2024-04-21T10:32:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff678bcc640 1 -- 172.21.15.26:0/1194468385 learned_addr learned my addr 172.21.15.26:0/1194468385 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff679bce640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff674105e30 0x7ff67419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff6793cd640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6741011a0 0x7ff674197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff678bcc640 1 -- 172.21.15.26:0/1194468385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff674105e30 msgr2=0x7ff67419be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff678bcc640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff674105e30 0x7ff67419be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff678bcc640 1 -- 172.21.15.26:0/1194468385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6741011a0 msgr2=0x7ff674197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff678bcc640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6741011a0 0x7ff674197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.511+0000 7ff678bcc640 1 -- 172.21.15.26:0/1194468385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67419c5c0 con 0x7ff674101b60 2024-04-21T10:32:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.512+0000 7ff678bcc640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff674101b60 0x7ff674197b60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff664009230 tx=0x7ff66402c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.512+0000 7ff66a7fc640 1 -- 172.21.15.26:0/1194468385 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664043020 con 0x7ff674101b60 2024-04-21T10:32:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.512+0000 7ff66a7fc640 1 -- 172.21.15.26:0/1194468385 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6640043d0 con 0x7ff674101b60 2024-04-21T10:32:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.512+0000 7ff67b631640 1 -- 172.21.15.26:0/1194468385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67419c850 con 0x7ff674101b60 2024-04-21T10:32:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.512+0000 7ff66a7fc640 1 -- 172.21.15.26:0/1194468385 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664003bc0 con 0x7ff674101b60 2024-04-21T10:32:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.512+0000 7ff67b631640 1 -- 172.21.15.26:0/1194468385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6741a4140 con 0x7ff674101b60 2024-04-21T10:32:02.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.513+0000 7ff67b631640 1 -- 172.21.15.26:0/1194468385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff674068de0 con 0x7ff674101b60 2024-04-21T10:32:02.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.514+0000 7ff66a7fc640 1 -- 172.21.15.26:0/1194468385 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7ff664046020 con 0x7ff674101b60 2024-04-21T10:32:02.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.515+0000 7ff66a7fc640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff6600a3f50 0x7ff6600a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.515+0000 7ff6793cd640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff6600a3f50 0x7ff6600a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.515+0000 7ff66a7fc640 1 -- 172.21.15.26:0/1194468385 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7ff6640f2ac0 con 0x7ff674101b60 2024-04-21T10:32:02.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.516+0000 7ff6793cd640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff6600a3f50 0x7ff6600a6410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff670005fd0 tx=0x7ff67000b390 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.520+0000 7ff66a7fc640 1 -- 172.21.15.26:0/1194468385 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff66404d020 con 0x7ff674101b60 2024-04-21T10:32:02.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.747+0000 7ff67b631640 1 -- 172.21.15.26:0/1194468385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff674109350 con 0x7ff674101b60 2024-04-21T10:32:02.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.748+0000 7ff66a7fc640 1 -- 172.21.15.26:0/1194468385 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11884 (secure 0 0 0) 0x7ff6640bf040 con 0x7ff674101b60 2024-04-21T10:32:02.748 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:32:02.748 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:30:56.186598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24478,24488]},"id":1}]} 2024-04-21T10:32:02.748 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 21 2024-04-21T10:32:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.750+0000 7ff67b631640 1 -- 172.21.15.26:0/1194468385 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff6600a3f50 msgr2=0x7ff6600a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.750+0000 7ff67b631640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff6600a3f50 0x7ff6600a6410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff670005fd0 tx=0x7ff67000b390 comp rx=0 tx=0).stop 2024-04-21T10:32:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.750+0000 7ff67b631640 1 -- 172.21.15.26:0/1194468385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff674101b60 msgr2=0x7ff674197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.750+0000 7ff67b631640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff674101b60 0x7ff674197b60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff664009230 tx=0x7ff66402c710 comp rx=0 tx=0).stop 2024-04-21T10:32:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.750+0000 7ff67b631640 1 -- 172.21.15.26:0/1194468385 shutdown_connections 2024-04-21T10:32:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.750+0000 7ff67b631640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7ff6600a3f50 0x7ff6600a6410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.750+0000 7ff67b631640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff674105e30 0x7ff67419be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.750+0000 7ff67b631640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff674101b60 0x7ff674197b60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.750+0000 7ff67b631640 1 --2- 172.21.15.26:0/1194468385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6741011a0 0x7ff674197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.750+0000 7ff67b631640 1 -- 172.21.15.26:0/1194468385 >> 172.21.15.26:0/1194468385 conn(0x7ff674077e30 msgr2=0x7ff6740fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.750+0000 7ff67b631640 1 -- 172.21.15.26:0/1194468385 shutdown_connections 2024-04-21T10:32:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.750+0000 7ff67b631640 1 -- 172.21.15.26:0/1194468385 wait complete. 2024-04-21T10:32:02.761 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --set-id dd4a255d --release --await' 2024-04-21T10:32:02.761 DEBUG:teuthology.orchestra.run.smithi026:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24496 quiesce db --set-id dd4a255d --release --await 2024-04-21T10:32:02.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.873+0000 7fd50ed50640 1 -- 172.21.15.26:0/1182695918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd508106170 msgr2=0x7fd50810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.873+0000 7fd50ed50640 1 --2- 172.21.15.26:0/1182695918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd508106170 0x7fd50810dd40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009900 tx=0x7fd4f002bc30 comp rx=0 tx=0).stop 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.873+0000 7fd50ed50640 1 -- 172.21.15.26:0/1182695918 shutdown_connections 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.873+0000 7fd50ed50640 1 --2- 172.21.15.26:0/1182695918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd508106170 0x7fd50810dd40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.873+0000 7fd50ed50640 1 --2- 172.21.15.26:0/1182695918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd508101a50 0x7fd508105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.873+0000 7fd50ed50640 1 --2- 172.21.15.26:0/1182695918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5081010a0 0x7fd508101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.873+0000 7fd50ed50640 1 -- 172.21.15.26:0/1182695918 >> 172.21.15.26:0/1182695918 conn(0x7fd5080771d0 msgr2=0x7fd5080775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.873+0000 7fd50ed50640 1 -- 172.21.15.26:0/1182695918 shutdown_connections 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.873+0000 7fd50ed50640 1 -- 172.21.15.26:0/1182695918 wait complete. 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.873+0000 7fd50ed50640 1 Processor -- start 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.873+0000 7fd50ed50640 1 -- start start 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.873+0000 7fd50ed50640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5081010a0 0x7fd508197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd50ed50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd508106170 0x7fd508197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd50ed50640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd50819bf00 0x7fd50819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd50ed50640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081110e0 con 0x7fd508106170 2024-04-21T10:32:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd50ed50640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd508110f60 con 0x7fd50819bf00 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd50ed50640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd508111260 con 0x7fd5081010a0 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd507fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd508106170 0x7fd508197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd507fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd508106170 0x7fd508197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33720/0 (socket says 172.21.15.26:33720) 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd507fff640 1 -- 172.21.15.26:0/1859667041 learned_addr learned my addr 172.21.15.26:0/1859667041 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd50caec640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5081010a0 0x7fd508197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd50d2ed640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd50819bf00 0x7fd50819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd507fff640 1 -- 172.21.15.26:0/1859667041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5081010a0 msgr2=0x7fd508197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd507fff640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5081010a0 0x7fd508197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd507fff640 1 -- 172.21.15.26:0/1859667041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd50819bf00 msgr2=0x7fd50819c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd507fff640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd50819bf00 0x7fd50819c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd507fff640 1 -- 172.21.15.26:0/1859667041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50819c9f0 con 0x7fd508106170 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd507fff640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd508106170 0x7fd508197b80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800eb90 tx=0x7fd4f800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd505ffb640 1 -- 172.21.15.26:0/1859667041 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f801e070 con 0x7fd508106170 2024-04-21T10:32:02.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd50ed50640 1 -- 172.21.15.26:0/1859667041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50819cce0 con 0x7fd508106170 2024-04-21T10:32:02.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd505ffb640 1 -- 172.21.15.26:0/1859667041 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f800ee90 con 0x7fd508106170 2024-04-21T10:32:02.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.874+0000 7fd505ffb640 1 -- 172.21.15.26:0/1859667041 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8019ae0 con 0x7fd508106170 2024-04-21T10:32:02.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.875+0000 7fd50ed50640 1 -- 172.21.15.26:0/1859667041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50806a810 con 0x7fd508106170 2024-04-21T10:32:02.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.877+0000 7fd505ffb640 1 -- 172.21.15.26:0/1859667041 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fd4f8021070 con 0x7fd508106170 2024-04-21T10:32:02.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.877+0000 7fd511a1c740 1 Processor -- start 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.877+0000 7fd511a1c740 1 -- start start 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd511a1c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6f310 0x557ce1c6f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd511a1c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c782d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd511a1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c7ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd511a1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557ce1bd4500 con 0x557ce1c78810 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd511a1c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557ce1b7a6a0 con 0x557ce1c6f310 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd511a1c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557ce1bd04c0 con 0x557ce1c6fc30 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd505ffb640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4f40a3ed0 0x7fd4f40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd507fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c7ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd50caec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c782d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd50d2ed640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4f40a3ed0 0x7fd4f40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd50caec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c782d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.26:48360/0 (socket says 172.21.15.26:48360) 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd50caec640 1 -- 172.21.15.26:0/2409512191 learned_addr learned my addr 172.21.15.26:0/2409512191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd50d2ed640 1 --2- 172.21.15.26:0/2409512191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6f310 0x557ce1c6f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd505ffb640 1 -- 172.21.15.26:0/1859667041 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fd4f80d2b30 con 0x7fd508106170 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd50caec640 1 -- 172.21.15.26:0/2409512191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6f310 msgr2=0x557ce1c6f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd50caec640 1 --2- 172.21.15.26:0/2409512191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6f310 0x557ce1c6f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd50caec640 1 -- 172.21.15.26:0/2409512191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 msgr2=0x557ce1c7ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd50caec640 1 --2- 172.21.15.26:0/2409512191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c7ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd50caec640 1 -- 172.21.15.26:0/2409512191 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ce1b995e0 con 0x557ce1c6fc30 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.878+0000 7fd507fff640 1 --2- 172.21.15.26:0/2409512191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c7ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.879+0000 7fd50d2ed640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4f40a3ed0 0x7fd4f40a6390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0005ab0 tx=0x7fd4f002c650 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.879+0000 7fd50caec640 1 --2- 172.21.15.26:0/2409512191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c782d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd500009d20 tx=0x7fd500011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5a2af012097d6f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.879+0000 7fd4feffd640 1 -- 172.21.15.26:0/2409512191 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500019620 con 0x557ce1c6fc30 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.879+0000 7fd4feffd640 1 -- 172.21.15.26:0/2409512191 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd500019c70 con 0x557ce1c6fc30 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.879+0000 7fd4feffd640 1 -- 172.21.15.26:0/2409512191 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500018b60 con 0x557ce1c6fc30 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 -- 172.21.15.26:0/2409512191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 msgr2=0x557ce1c782d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 --2- 172.21.15.26:0/2409512191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c782d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd500009d20 tx=0x7fd500011810 comp rx=0 tx=0).stop 2024-04-21T10:32:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 -- 172.21.15.26:0/2409512191 shutdown_connections 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 --2- 172.21.15.26:0/2409512191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c7ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 --2- 172.21.15.26:0/2409512191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c782d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 --2- 172.21.15.26:0/2409512191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6f310 0x557ce1c6f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 -- 172.21.15.26:0/2409512191 >> 172.21.15.26:0/2409512191 conn(0x557ce1c69a40 msgr2=0x557ce1c69e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 -- 172.21.15.26:0/2409512191 shutdown_connections 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 -- 172.21.15.26:0/2409512191 wait complete. 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 -- start start 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 Processor -- start 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c6f310 0x557ce1c78270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c75600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.880+0000 7fd511a1c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c78810 0x557ce1c75d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd511a1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557ce1c7b600 con 0x557ce1c6f310 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd511a1c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557ce1b8da60 con 0x557ce1c78810 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd511a1c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x557ce1c7b780 con 0x557ce1c6fc30 2024-04-21T10:32:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd511a1c740 10 client.0 _ll_register_callbacks cb 0x557ce18e6110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd50caec640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c6f310 0x557ce1c78270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd50caec640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c6f310 0x557ce1c78270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33748/0 (socket says 172.21.15.26:33748) 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd50caec640 1 -- 172.21.15.26:0/266670636 learned_addr learned my addr 172.21.15.26:0/266670636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd507fff640 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c75600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd50d2ed640 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c78810 0x557ce1c75d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd507fff640 1 -- 172.21.15.26:0/266670636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c78810 msgr2=0x557ce1c75d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd507fff640 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c78810 0x557ce1c75d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd507fff640 1 -- 172.21.15.26:0/266670636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c6f310 msgr2=0x557ce1c78270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd507fff640 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c6f310 0x557ce1c78270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd507fff640 1 -- 172.21.15.26:0/266670636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ce18e6920 con 0x557ce1c6fc30 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd50caec640 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c6f310 0x557ce1c78270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd507fff640 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c75600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80d0160 tx=0x7fd4f806e340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f26ba0d52671202 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd4e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-04-21T10:32:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd4e7fff640 1 -- 172.21.15.26:0/266670636 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f809ea60 con 0x557ce1c6fc30 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd511a1c740 1 -- 172.21.15.26:0/266670636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd500019c70 con 0x557ce1c6fc30 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.881+0000 7fd4e7fff640 1 -- 172.21.15.26:0/266670636 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fd4f806d1b0 con 0x557ce1c6fc30 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.882+0000 7fd4e7fff640 1 -- 172.21.15.26:0/266670636 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd4f80a2d60 con 0x557ce1c6fc30 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.882+0000 7fd511a1c740 10 client.25309 fetch_fsmap learned FSMap version 21 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.882+0000 7fd511a1c740 1 -- 172.21.15.26:0/266670636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557ce19b2d30 con 0x557ce1c6fc30 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.882+0000 7fd4e7fff640 1 -- 172.21.15.26:0/266670636 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd500019c70 con 0x557ce1c6fc30 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.882+0000 7fd4e7fff640 1 -- 172.21.15.26:0/266670636 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd4f809fce0 con 0x557ce1c6fc30 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.882+0000 7fd511a1c740 10 client.25309 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.882+0000 7fd511a1c740 10 client.25309 resolve_mds: validated gid 24496 aka daemon mds.i 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.882+0000 7fd511a1c740 20 client.25309 populate_metadata read hostname 'smithi026' 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.882+0000 7fd511a1c740 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x557ce1c88960 0x557ce1c8adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.882+0000 7fd511a1c740 4 client.25309 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.882+0000 7fd511a1c740 1 -- 172.21.15.26:0/266670636 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ce1ba7a10 con 0x557ce1c88960 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.882+0000 7fd50caec640 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x557ce1c88960 0x557ce1c8adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.883+0000 7fd50caec640 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x557ce1c88960 0x557ce1c8adb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b29328786c4220b2 server_cookie=33699ba6e72e2b4f in_seq=0 out_seq=0 2024-04-21T10:32:02.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.883+0000 7fd4e7fff640 10 client.25309 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.883+0000 7fd4e7fff640 1 -- 172.21.15.26:0/266670636 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ce1ba7a10 con 0x557ce1c88960 2024-04-21T10:32:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.883+0000 7fd4e7fff640 10 client.25309 handle_command_reply: tid=0 2024-04-21T10:32:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.883+0000 7fd511a1c740 1 client.25309 shutdown 2024-04-21T10:32:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.883+0000 7fd511a1c740 1 -- 172.21.15.26:0/266670636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 msgr2=0x557ce1c75600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.883+0000 7fd511a1c740 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c75600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80d0160 tx=0x7fd4f806e340 comp rx=0 tx=0).stop 2024-04-21T10:32:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.884+0000 7fd511a1c740 1 -- 172.21.15.26:0/266670636 shutdown_connections 2024-04-21T10:32:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.884+0000 7fd511a1c740 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x557ce1c88960 0x557ce1c8adb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.884+0000 7fd511a1c740 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c78810 0x557ce1c75d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.884+0000 7fd511a1c740 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c75600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.884+0000 7fd4e7fff640 0 client.25309 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.884+0000 7fd511a1c740 1 --2- 172.21.15.26:0/266670636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c6f310 0x557ce1c78270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.884+0000 7fd511a1c740 1 -- 172.21.15.26:0/266670636 >> 172.21.15.26:0/266670636 conn(0x557ce1c69a40 msgr2=0x557ce1c6b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:02.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.884+0000 7fd511a1c740 1 -- 172.21.15.26:0/266670636 shutdown_connections 2024-04-21T10:32:02.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.884+0000 7fd511a1c740 1 -- 172.21.15.26:0/266670636 wait complete. 2024-04-21T10:32:02.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.884+0000 7fd511a1c740 20 client.25309 trim_cache size 0 max 16384 2024-04-21T10:32:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd511a1c740 1 Processor -- start 2024-04-21T10:32:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd511a1c740 1 -- start start 2024-04-21T10:32:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd511a1c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6f310 0x557ce1c74580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd511a1c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c90650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd511a1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c90b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd511a1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081ae1d0 con 0x557ce1c78810 2024-04-21T10:32:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd511a1c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081abaa0 con 0x557ce1c6f310 2024-04-21T10:32:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd511a1c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081a98f0 con 0x557ce1c6fc30 2024-04-21T10:32:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd50d2ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c90b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd50d2ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c90b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33758/0 (socket says 172.21.15.26:33758) 2024-04-21T10:32:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd50d2ed640 1 -- 172.21.15.26:0/2697797632 learned_addr learned my addr 172.21.15.26:0/2697797632 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd507fff640 1 --2- 172.21.15.26:0/2697797632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c90650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.901+0000 7fd50caec640 1 --2- 172.21.15.26:0/2697797632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6f310 0x557ce1c74580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.902+0000 7fd507fff640 1 -- 172.21.15.26:0/2697797632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6f310 msgr2=0x557ce1c74580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.902+0000 7fd507fff640 1 --2- 172.21.15.26:0/2697797632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6f310 0x557ce1c74580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.902+0000 7fd507fff640 1 -- 172.21.15.26:0/2697797632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 msgr2=0x557ce1c90b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.902+0000 7fd507fff640 1 --2- 172.21.15.26:0/2697797632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c90b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.902+0000 7fd507fff640 1 -- 172.21.15.26:0/2697797632 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ce1c875b0 con 0x557ce1c6fc30 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.902+0000 7fd50d2ed640 1 --2- 172.21.15.26:0/2697797632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c90b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.902+0000 7fd50caec640 1 --2- 172.21.15.26:0/2697797632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6f310 0x557ce1c74580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.902+0000 7fd507fff640 1 --2- 172.21.15.26:0/2697797632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c90650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800cba0 tx=0x7fd4f8009a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd76aad07de2ae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.902+0000 7fd4fe7fc640 1 -- 172.21.15.26:0/2697797632 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f809ea60 con 0x557ce1c6fc30 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.902+0000 7fd4fe7fc640 1 -- 172.21.15.26:0/2697797632 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f80a7620 con 0x557ce1c6fc30 2024-04-21T10:32:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.902+0000 7fd4fe7fc640 1 -- 172.21.15.26:0/2697797632 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f809e830 con 0x557ce1c6fc30 2024-04-21T10:32:02.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.903+0000 7fd511a1c740 1 -- 172.21.15.26:0/2697797632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 msgr2=0x557ce1c90650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.903+0000 7fd511a1c740 1 --2- 172.21.15.26:0/2697797632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c90650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800cba0 tx=0x7fd4f8009a60 comp rx=0 tx=0).stop 2024-04-21T10:32:02.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.903+0000 7fd511a1c740 1 -- 172.21.15.26:0/2697797632 shutdown_connections 2024-04-21T10:32:02.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.903+0000 7fd511a1c740 1 --2- 172.21.15.26:0/2697797632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c90b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.903+0000 7fd511a1c740 1 --2- 172.21.15.26:0/2697797632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6fc30 0x557ce1c90650 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.903+0000 7fd511a1c740 1 --2- 172.21.15.26:0/2697797632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6f310 0x557ce1c74580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.903+0000 7fd511a1c740 1 -- 172.21.15.26:0/2697797632 >> 172.21.15.26:0/2697797632 conn(0x557ce1c69a40 msgr2=0x557ce1c68b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:02.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.903+0000 7fd511a1c740 1 -- 172.21.15.26:0/2697797632 shutdown_connections 2024-04-21T10:32:02.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.903+0000 7fd511a1c740 1 -- 172.21.15.26:0/2697797632 wait complete. 2024-04-21T10:32:02.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.903+0000 7fd511a1c740 1 -- start start 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.903+0000 7fd511a1c740 1 Processor -- start 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd511a1c740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6f310 0x557ce1c80bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd511a1c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6fc30 0x557ce1c7b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd511a1c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c7bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd511a1c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081a9f60 con 0x557ce1c78810 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd511a1c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081ab3d0 con 0x557ce1c6fc30 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd511a1c740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd5081aa630 con 0x557ce1c6f310 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd511a1c740 10 client.0 _ll_register_callbacks cb 0x7fd4f80099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd50d2ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c7bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd50d2ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c7bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33768/0 (socket says 172.21.15.26:33768) 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd50d2ed640 1 -- 172.21.15.26:0/3029510488 learned_addr learned my addr 172.21.15.26:0/3029510488 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd50caec640 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6f310 0x557ce1c80bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd507fff640 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6fc30 0x557ce1c7b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd50caec640 1 -- 172.21.15.26:0/3029510488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6fc30 msgr2=0x557ce1c7b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd50caec640 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6fc30 0x557ce1c7b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd50caec640 1 -- 172.21.15.26:0/3029510488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 msgr2=0x557ce1c7bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd50caec640 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c7bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd50caec640 1 -- 172.21.15.26:0/3029510488 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ce1c7c410 con 0x557ce1c6f310 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd507fff640 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6fc30 0x557ce1c7b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.904+0000 7fd50d2ed640 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c7bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd50caec640 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6f310 0x557ce1c80bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd500011d10 tx=0x7fd500011dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb6e1893046920f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd4e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd4e7fff640 1 -- 172.21.15.26:0/3029510488 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5000185d0 con 0x557ce1c6f310 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd511a1c740 1 -- 172.21.15.26:0/3029510488 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4f80a7620 con 0x557ce1c6f310 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd4e7fff640 1 -- 172.21.15.26:0/3029510488 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fd5000053c0 con 0x557ce1c6f310 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd4e7fff640 1 -- 172.21.15.26:0/3029510488 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd50003a050 con 0x557ce1c6f310 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd511a1c740 10 client.25315 fetch_fsmap learned FSMap version 21 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd511a1c740 1 -- 172.21.15.26:0/3029510488 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557ce1cefb90 con 0x557ce1c6f310 2024-04-21T10:32:02.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd4e7fff640 1 -- 172.21.15.26:0/3029510488 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f80a7620 con 0x557ce1c6f310 2024-04-21T10:32:02.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd4e7fff640 1 -- 172.21.15.26:0/3029510488 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd50002b6b0 con 0x557ce1c6f310 2024-04-21T10:32:02.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd511a1c740 10 client.25315 fetch_fsmap finished waiting for FSMap version 21 2024-04-21T10:32:02.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd511a1c740 10 client.25315 resolve_mds: validated gid 24496 aka daemon mds.i 2024-04-21T10:32:02.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd511a1c740 20 client.25315 populate_metadata read hostname 'smithi026' 2024-04-21T10:32:02.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd511a1c740 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x557ce1cf00c0 0x557ce1cf24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:02.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd511a1c740 4 client.25315 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dd4a255d", "release": true, "await": true}] 2024-04-21T10:32:02.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.905+0000 7fd511a1c740 1 -- 172.21.15.26:0/3029510488 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dd4a255d", "release": true, "await": true}) -- 0x557ce1be1f20 con 0x557ce1cf00c0 2024-04-21T10:32:02.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.906+0000 7fd507fff640 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x557ce1cf00c0 0x557ce1cf24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:02.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.906+0000 7fd507fff640 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x557ce1cf00c0 0x557ce1cf24b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed6cd2b7d14cd6dd server_cookie=6072362152b833b6 in_seq=0 out_seq=0 2024-04-21T10:32:02.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.906+0000 7fd4e7fff640 10 client.25315 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd4e7fff640 1 -- 172.21.15.26:0/3029510488 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x557ce1be1f20 con 0x557ce1cf00c0 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd4e7fff640 10 client.25315 handle_command_reply: tid=0 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd511a1c740 1 client.25315 shutdown 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd511a1c740 1 -- 172.21.15.26:0/3029510488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6f310 msgr2=0x557ce1c80bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd511a1c740 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6f310 0x557ce1c80bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd500011d10 tx=0x7fd500011dc0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd511a1c740 1 -- 172.21.15.26:0/3029510488 shutdown_connections 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd511a1c740 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x557ce1cf00c0 0x557ce1cf24b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd511a1c740 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ce1c78810 0x557ce1c7bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd4e7fff640 0 client.25315 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd511a1c740 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557ce1c6fc30 0x557ce1c7b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd511a1c740 1 --2- 172.21.15.26:0/3029510488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557ce1c6f310 0x557ce1c80bd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd511a1c740 1 -- 172.21.15.26:0/3029510488 >> 172.21.15.26:0/3029510488 conn(0x557ce1c69a40 msgr2=0x557ce1c6abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd511a1c740 1 -- 172.21.15.26:0/3029510488 shutdown_connections 2024-04-21T10:32:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd511a1c740 1 -- 172.21.15.26:0/3029510488 wait complete. 2024-04-21T10:32:02.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.911+0000 7fd511a1c740 20 client.25315 trim_cache size 0 max 16384 2024-04-21T10:32:02.913 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:32:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 21, 2024-04-21T10:32:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24496, 2024-04-21T10:32:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 8, 2024-04-21T10:32:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": { 2024-04-21T10:32:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: "dd4a255d": { 2024-04-21T10:32:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 8, 2024-04-21T10:32:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: "age_ref": 0.0, 2024-04-21T10:32:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-04-21T10:32:02.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: "timeout": 30.0, 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: "expiration": 79.0, 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: "members": { 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: "file:/": { 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: "excluded": false, 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: "state": { 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RELEASED", 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: "age": 0.0 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:32:02.914 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:32:02.915 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:32:02.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.912+0000 7fd50ed50640 1 -- 172.21.15.26:0/1859667041 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4f40a3ed0 msgr2=0x7fd4f40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.912+0000 7fd50ed50640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4f40a3ed0 0x7fd4f40a6390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0005ab0 tx=0x7fd4f002c650 comp rx=0 tx=0).stop 2024-04-21T10:32:02.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.912+0000 7fd50ed50640 1 -- 172.21.15.26:0/1859667041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd508106170 msgr2=0x7fd508197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:02.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.912+0000 7fd50ed50640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd508106170 0x7fd508197b80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800eb90 tx=0x7fd4f800c6a0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.912+0000 7fd50ed50640 1 -- 172.21.15.26:0/1859667041 shutdown_connections 2024-04-21T10:32:02.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.912+0000 7fd50ed50640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4f40a3ed0 0x7fd4f40a6390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.912+0000 7fd50ed50640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd50819bf00 0x7fd50819c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.912+0000 7fd50ed50640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd508106170 0x7fd508197b80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.912+0000 7fd50ed50640 1 --2- 172.21.15.26:0/1859667041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5081010a0 0x7fd508197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:02.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.912+0000 7fd50ed50640 1 -- 172.21.15.26:0/1859667041 >> 172.21.15.26:0/1859667041 conn(0x7fd5080771d0 msgr2=0x7fd5080fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:02.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.912+0000 7fd50ed50640 1 -- 172.21.15.26:0/1859667041 shutdown_connections 2024-04-21T10:32:02.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:02.912+0000 7fd50ed50640 1 -- 172.21.15.26:0/1859667041 wait complete. 2024-04-21T10:32:02.917 INFO:tasks.workunit.client.0.smithi026.stderr:+ set -e 2024-04-21T10:32:02.917 INFO:tasks.workunit.client.0.smithi026.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-21T10:32:02.917 INFO:tasks.workunit.client.0.smithi026.stderr:Cloning into 'xfstests-dev'... 2024-04-21T10:32:02.922 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: dd4a255d, seconds elapsed: 20.0 2024-04-21T10:32:03.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:03 smithi026 ceph-mon[27965]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-21T10:32:03.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:32:03.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:03 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1194468385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:32:03.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:32:03.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:03 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:32:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:03 smithi169 ceph-mon[31469]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-21T10:32:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:32:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:03 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1194468385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:32:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:32:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:03 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:32:03.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:03 smithi190 ceph-mon[31423]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-21T10:32:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:32:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:03 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1194468385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:32:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:32:03.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:03 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:32:04.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:04 smithi026 ceph-mon[27965]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:32:04.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:04 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:32:04.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:04 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:32:04.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:04 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:32:04.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:04 smithi169 ceph-mon[31469]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:32:04.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:04 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:32:04.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:04 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:32:04.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:04 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:32:04.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:04 smithi190 ceph-mon[31423]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:32:04.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:04 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:32:04.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:04 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:32:04.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:04 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:32:05.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:05 smithi026 ceph-mon[27965]: pgmap v36: 97 pgs: 97 active+clean; 1.5 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-04-21T10:32:05.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:05 smithi169 ceph-mon[31469]: pgmap v36: 97 pgs: 97 active+clean; 1.5 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-04-21T10:32:05.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:05 smithi190 ceph-mon[31423]: pgmap v36: 97 pgs: 97 active+clean; 1.5 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-04-21T10:32:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.427+0000 7fd4e1985640 1 -- 172.21.15.26:0/1420038001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4dc107290 msgr2=0x7fd4dc10ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.427+0000 7fd4e1985640 1 --2- 172.21.15.26:0/1420038001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4dc107290 0x7fd4dc10ee60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009ca0 tx=0x7fd4d402c0f0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.427+0000 7fd4e1985640 1 -- 172.21.15.26:0/1420038001 shutdown_connections 2024-04-21T10:32:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.427+0000 7fd4e1985640 1 --2- 172.21.15.26:0/1420038001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4dc107290 0x7fd4dc10ee60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.427+0000 7fd4e1985640 1 --2- 172.21.15.26:0/1420038001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc102b70 0x7fd4dc106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.427+0000 7fd4e1985640 1 --2- 172.21.15.26:0/1420038001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc1021c0 0x7fd4dc1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.427+0000 7fd4e1985640 1 -- 172.21.15.26:0/1420038001 >> 172.21.15.26:0/1420038001 conn(0x7fd4dc0fda20 msgr2=0x7fd4dc0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.428+0000 7fd4e1985640 1 -- 172.21.15.26:0/1420038001 shutdown_connections 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.428+0000 7fd4e1985640 1 -- 172.21.15.26:0/1420038001 wait complete. 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.428+0000 7fd4e1985640 1 Processor -- start 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.428+0000 7fd4e1985640 1 -- start start 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.428+0000 7fd4e1985640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc1021c0 0x7fd4dc198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.428+0000 7fd4e1985640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4dc107290 0x7fd4dc198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.428+0000 7fd4e1985640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc19cfb0 0x7fd4dc19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.428+0000 7fd4e1985640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc112200 con 0x7fd4dc107290 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.428+0000 7fd4e1985640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc112080 con 0x7fd4dc19cfb0 2024-04-21T10:32:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.428+0000 7fd4e1985640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc112380 con 0x7fd4dc1021c0 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4dc107290 0x7fd4dc198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4dc107290 0x7fd4dc198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40624/0 (socket says 172.21.15.26:40624) 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4d3fff640 1 -- 172.21.15.26:0/1639786895 learned_addr learned my addr 172.21.15.26:0/1639786895 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4daffd640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc1021c0 0x7fd4dc198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4db7fe640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc19cfb0 0x7fd4dc19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4d3fff640 1 -- 172.21.15.26:0/1639786895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc1021c0 msgr2=0x7fd4dc198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4d3fff640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc1021c0 0x7fd4dc198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4d3fff640 1 -- 172.21.15.26:0/1639786895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc19cfb0 msgr2=0x7fd4dc19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4d3fff640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc19cfb0 0x7fd4dc19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4d3fff640 1 -- 172.21.15.26:0/1639786895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4dc19daa0 con 0x7fd4dc107290 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4d3fff640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4dc107290 0x7fd4dc198c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400d740 tx=0x7fd4c400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4e0983640 1 -- 172.21.15.26:0/1639786895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4017070 con 0x7fd4dc107290 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4e0983640 1 -- 172.21.15.26:0/1639786895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c400de50 con 0x7fd4dc107290 2024-04-21T10:32:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4e1985640 1 -- 172.21.15.26:0/1639786895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4dc19dd90 con 0x7fd4dc107290 2024-04-21T10:32:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4e0983640 1 -- 172.21.15.26:0/1639786895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4004ce0 con 0x7fd4dc107290 2024-04-21T10:32:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.429+0000 7fd4e1985640 1 -- 172.21.15.26:0/1639786895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4dc1a5580 con 0x7fd4dc107290 2024-04-21T10:32:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.432+0000 7fd4e0983640 1 -- 172.21.15.26:0/1639786895 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fd4c4004e80 con 0x7fd4dc107290 2024-04-21T10:32:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.432+0000 7fd4e4574740 1 Processor -- start 2024-04-21T10:32:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.432+0000 7fd4e4574740 1 -- start start 2024-04-21T10:32:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.432+0000 7fd4e4574740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa38d0 0x55b355aa2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.432+0000 7fd4e4574740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa2b60 0x55b355aa2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.432+0000 7fd4e4574740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 0x55b355aae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.432+0000 7fd4e4574740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b355a078d0 con 0x55b355aac0e0 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.432+0000 7fd4e4574740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b3559a4e80 con 0x55b355aa2b60 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.432+0000 7fd4e4574740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b355a03890 con 0x55b355aa38d0 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 0x55b355aae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4e0983640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4c80a3ff0 0x7fd4c80a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 0x55b355aae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40636/0 (socket says 172.21.15.26:40636) 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4d3fff640 1 -- 172.21.15.26:0/1675733766 learned_addr learned my addr 172.21.15.26:0/1675733766 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4db7fe640 1 --2- 172.21.15.26:0/1675733766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa38d0 0x55b355aa2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4daffd640 1 --2- 172.21.15.26:0/1675733766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa2b60 0x55b355aa2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4db7fe640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4c80a3ff0 0x7fd4c80a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4e0983640 1 -- 172.21.15.26:0/1639786895 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fd4c40d02c0 con 0x7fd4dc107290 2024-04-21T10:32:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4d3fff640 1 -- 172.21.15.26:0/1675733766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa38d0 msgr2=0x55b355aa2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4d3fff640 1 --2- 172.21.15.26:0/1675733766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa38d0 0x55b355aa2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4d3fff640 1 -- 172.21.15.26:0/1675733766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa2b60 msgr2=0x55b355aa2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4d3fff640 1 --2- 172.21.15.26:0/1675733766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa2b60 0x55b355aa2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4d3fff640 1 -- 172.21.15.26:0/1675733766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3559c6320 con 0x55b355aac0e0 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4db7fe640 1 --2- 172.21.15.26:0/1675733766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa38d0 0x55b355aa2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4d3fff640 1 --2- 172.21.15.26:0/1675733766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 0x55b355aae4d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40cd780 tx=0x7fd4c4004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57953a26ade624b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4db7fe640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4c80a3ff0 0x7fd4c80a64b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4004af0 tx=0x7fd4d4005c50 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4d1ffb640 1 -- 172.21.15.26:0/1675733766 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4020070 con 0x55b355aac0e0 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.433+0000 7fd4d1ffb640 1 -- 172.21.15.26:0/1675733766 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c40a03e0 con 0x55b355aac0e0 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.434+0000 7fd4d1ffb640 1 -- 172.21.15.26:0/1675733766 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40cdaf0 con 0x55b355aac0e0 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.434+0000 7fd4e4574740 1 -- 172.21.15.26:0/1675733766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 msgr2=0x55b355aae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.434+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1675733766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 0x55b355aae4d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40cd780 tx=0x7fd4c4004780 comp rx=0 tx=0).stop 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.434+0000 7fd4e4574740 1 -- 172.21.15.26:0/1675733766 shutdown_connections 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.434+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1675733766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 0x55b355aae4d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.434+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1675733766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa2b60 0x55b355aa2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.434+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1675733766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa38d0 0x55b355aa2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.434+0000 7fd4e4574740 1 -- 172.21.15.26:0/1675733766 >> 172.21.15.26:0/1675733766 conn(0x55b355a69c90 msgr2=0x55b355a9d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.434+0000 7fd4e4574740 1 -- 172.21.15.26:0/1675733766 shutdown_connections 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.434+0000 7fd4e4574740 1 -- 172.21.15.26:0/1675733766 wait complete. 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4e4574740 1 -- start start 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4e4574740 1 Processor -- start 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4e4574740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa2b60 0x55b355aaba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4e4574740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa38d0 0x55b355aa8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4e4574740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 0x55b355aa92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4e4574740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b355aaedb0 con 0x55b355aac0e0 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4e4574740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b35598feb0 con 0x55b355aa2b60 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4e4574740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b355aaef30 con 0x55b355aa38d0 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4e4574740 10 client.0 _ll_register_callbacks cb 0x55b3559bcb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:32:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 0x55b355aa92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4db7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 0x55b355aa92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40640/0 (socket says 172.21.15.26:40640) 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4db7fe640 1 -- 172.21.15.26:0/1798402845 learned_addr learned my addr 172.21.15.26:0/1798402845 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.435+0000 7fd4daffd640 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa2b60 0x55b355aaba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4d3fff640 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa38d0 0x55b355aa8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4db7fe640 1 -- 172.21.15.26:0/1798402845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa38d0 msgr2=0x55b355aa8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4db7fe640 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa38d0 0x55b355aa8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4db7fe640 1 -- 172.21.15.26:0/1798402845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa2b60 msgr2=0x55b355aaba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4db7fe640 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa2b60 0x55b355aaba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4db7fe640 1 -- 172.21.15.26:0/1798402845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b35571a210 con 0x55b355aac0e0 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4db7fe640 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 0x55b355aa92d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009ca0 tx=0x7fd4d40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616fb57fd42c618b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4baffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4baffd640 1 -- 172.21.15.26:0/1798402845 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4052060 con 0x55b355aac0e0 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4baffd640 1 -- 172.21.15.26:0/1798402845 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fd4d4003e00 con 0x55b355aac0e0 2024-04-21T10:32:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4e4574740 1 -- 172.21.15.26:0/1798402845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c40a03e0 con 0x55b355aac0e0 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4baffd640 1 -- 172.21.15.26:0/1798402845 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd4d4039040 con 0x55b355aac0e0 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4e4574740 10 client.15744 fetch_fsmap learned FSMap version 22 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4e4574740 1 -- 172.21.15.26:0/1798402845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b3557e6620 con 0x55b355aac0e0 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.436+0000 7fd4baffd640 1 -- 172.21.15.26:0/1798402845 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c40a03e0 con 0x55b355aac0e0 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.437+0000 7fd4baffd640 1 -- 172.21.15.26:0/1798402845 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fd4d40347a0 con 0x55b355aac0e0 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.437+0000 7fd4e4574740 10 client.15744 fetch_fsmap finished waiting for FSMap version 22 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.437+0000 7fd4e4574740 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.437+0000 7fd4e4574740 20 client.15744 populate_metadata read hostname 'smithi026' 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.437+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55b355abc000 0x55b355abe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.437+0000 7fd4e4574740 4 client.15744 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.437+0000 7fd4e4574740 1 -- 172.21.15.26:0/1798402845 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b355a1b880 con 0x55b355abc000 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.437+0000 7fd4daffd640 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55b355abc000 0x55b355abe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.437+0000 7fd4baffd640 10 client.15744 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.437+0000 7fd4daffd640 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55b355abc000 0x55b355abe410 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfbee0739a0b0f68 server_cookie=c9704c32b1773d36 in_seq=0 out_seq=0 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.438+0000 7fd4baffd640 1 -- 172.21.15.26:0/1798402845 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b355a1b880 con 0x55b355abc000 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.438+0000 7fd4baffd640 10 client.15744 handle_command_reply: tid=0 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.438+0000 7fd4e4574740 1 client.15744 shutdown 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.438+0000 7fd4e4574740 1 -- 172.21.15.26:0/1798402845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 msgr2=0x55b355aa92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.438+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 0x55b355aa92d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009ca0 tx=0x7fd4d40095c0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.439+0000 7fd4e4574740 1 -- 172.21.15.26:0/1798402845 shutdown_connections 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.439+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55b355abc000 0x55b355abe410 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.439+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aac0e0 0x55b355aa92d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.439+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa38d0 0x55b355aa8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.439+0000 7fd4baffd640 0 client.15744 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.439+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1798402845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa2b60 0x55b355aaba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.439+0000 7fd4e4574740 1 -- 172.21.15.26:0/1798402845 >> 172.21.15.26:0/1798402845 conn(0x55b355a69c90 msgr2=0x55b355a9f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.439+0000 7fd4e4574740 1 -- 172.21.15.26:0/1798402845 shutdown_connections 2024-04-21T10:32:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.439+0000 7fd4e4574740 1 -- 172.21.15.26:0/1798402845 wait complete. 2024-04-21T10:32:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.439+0000 7fd4e4574740 20 client.15744 trim_cache size 0 max 16384 2024-04-21T10:32:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.454+0000 7fd4e4574740 1 Processor -- start 2024-04-21T10:32:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.454+0000 7fd4e4574740 1 -- start start 2024-04-21T10:32:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.454+0000 7fd4e4574740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa2b60 0x55b355ab7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.454+0000 7fd4e4574740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa38d0 0x55b355aab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.454+0000 7fd4e4574740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aac0e0 0x55b355aaba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.454+0000 7fd4e4574740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc1b03f0 con 0x55b355aa2b60 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.454+0000 7fd4e4574740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc1adcc0 con 0x55b355aa38d0 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.454+0000 7fd4e4574740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc1abb30 con 0x55b355aac0e0 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.454+0000 7fd4daffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa2b60 0x55b355ab7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.454+0000 7fd4daffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa2b60 0x55b355ab7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40656/0 (socket says 172.21.15.26:40656) 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4daffd640 1 -- 172.21.15.26:0/1334615480 learned_addr learned my addr 172.21.15.26:0/1334615480 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4db7fe640 1 --2- 172.21.15.26:0/1334615480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aac0e0 0x55b355aaba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4d3fff640 1 --2- 172.21.15.26:0/1334615480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa38d0 0x55b355aab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4daffd640 1 -- 172.21.15.26:0/1334615480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aac0e0 msgr2=0x55b355aaba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4daffd640 1 --2- 172.21.15.26:0/1334615480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aac0e0 0x55b355aaba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4daffd640 1 -- 172.21.15.26:0/1334615480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa38d0 msgr2=0x55b355aab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4daffd640 1 --2- 172.21.15.26:0/1334615480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa38d0 0x55b355aab4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4daffd640 1 -- 172.21.15.26:0/1334615480 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b355abea90 con 0x55b355aa2b60 2024-04-21T10:32:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4d3fff640 1 --2- 172.21.15.26:0/1334615480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa38d0 0x55b355aab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4daffd640 1 --2- 172.21.15.26:0/1334615480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa2b60 0x55b355ab7cb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55b355abecc0 tx=0x7fd4cc00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ca4852717d8176 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4d17fa640 1 -- 172.21.15.26:0/1334615480 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc004220 con 0x55b355aa2b60 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4d17fa640 1 -- 172.21.15.26:0/1334615480 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4cc0043c0 con 0x55b355aa2b60 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.455+0000 7fd4d17fa640 1 -- 172.21.15.26:0/1334615480 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc020230 con 0x55b355aa2b60 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.456+0000 7fd4e4574740 1 -- 172.21.15.26:0/1334615480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa2b60 msgr2=0x55b355ab7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.456+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1334615480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa2b60 0x55b355ab7cb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55b355abecc0 tx=0x7fd4cc00b410 comp rx=0 tx=0).stop 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.456+0000 7fd4e4574740 1 -- 172.21.15.26:0/1334615480 shutdown_connections 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.456+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1334615480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aac0e0 0x55b355aaba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.456+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1334615480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aa38d0 0x55b355aab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.456+0000 7fd4e4574740 1 --2- 172.21.15.26:0/1334615480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa2b60 0x55b355ab7cb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.456+0000 7fd4e4574740 1 -- 172.21.15.26:0/1334615480 >> 172.21.15.26:0/1334615480 conn(0x55b355a69c90 msgr2=0x55b355a9d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.456+0000 7fd4e4574740 1 -- 172.21.15.26:0/1334615480 shutdown_connections 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.456+0000 7fd4e4574740 1 -- 172.21.15.26:0/1334615480 wait complete. 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.456+0000 7fd4e4574740 1 -- start start 2024-04-21T10:32:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.456+0000 7fd4e4574740 1 Processor -- start 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4e4574740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa2b60 0x55b355ab41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4e4574740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa38d0 0x55b355ab4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4e4574740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aac0e0 0x55b355ab1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4e4574740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc1ac180 con 0x55b355aa38d0 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4e4574740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc1ad5f0 con 0x55b355aac0e0 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4e4574740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc1ac850 con 0x55b355aa2b60 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4e4574740 10 client.0 _ll_register_callbacks cb 0x7fd4d4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa38d0 0x55b355ab4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa38d0 0x55b355ab4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40664/0 (socket says 172.21.15.26:40664) 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4d3fff640 1 -- 172.21.15.26:0/415332817 learned_addr learned my addr 172.21.15.26:0/415332817 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4db7fe640 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aac0e0 0x55b355ab1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4daffd640 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa2b60 0x55b355ab41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4d3fff640 1 -- 172.21.15.26:0/415332817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa2b60 msgr2=0x55b355ab41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4d3fff640 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa2b60 0x55b355ab41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4d3fff640 1 -- 172.21.15.26:0/415332817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aac0e0 msgr2=0x55b355ab1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4d3fff640 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aac0e0 0x55b355ab1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4d3fff640 1 -- 172.21.15.26:0/415332817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b355aa1e30 con 0x55b355aa38d0 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4d3fff640 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa38d0 0x55b355ab4700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd4c409bde0 tx=0x7fd4c40983f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3d2090db564e9e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4baffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4baffd640 1 -- 172.21.15.26:0/415332817 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4020040 con 0x55b355aa38d0 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4baffd640 1 -- 172.21.15.26:0/415332817 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fd4c40d0040 con 0x55b355aa38d0 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.457+0000 7fd4e4574740 1 -- 172.21.15.26:0/415332817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4cc0043c0 con 0x55b355aa38d0 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.458+0000 7fd4baffd640 1 -- 172.21.15.26:0/415332817 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd4c40a03e0 con 0x55b355aa38d0 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.458+0000 7fd4e4574740 10 client.15750 fetch_fsmap learned FSMap version 22 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.458+0000 7fd4e4574740 1 -- 172.21.15.26:0/415332817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b355b22d40 con 0x55b355aa38d0 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.458+0000 7fd4baffd640 1 -- 172.21.15.26:0/415332817 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4cc0043c0 con 0x55b355aa38d0 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.458+0000 7fd4baffd640 1 -- 172.21.15.26:0/415332817 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fd4c4076030 con 0x55b355aa38d0 2024-04-21T10:32:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.458+0000 7fd4e4574740 10 client.15750 fetch_fsmap finished waiting for FSMap version 22 2024-04-21T10:32:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.458+0000 7fd4e4574740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:32:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.458+0000 7fd4e4574740 20 client.15750 populate_metadata read hostname 'smithi026' 2024-04-21T10:32:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.458+0000 7fd4e4574740 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55b355b230f0 0x55b355b25510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.458+0000 7fd4e4574740 4 client.15750 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T10:32:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.458+0000 7fd4e4574740 1 -- 172.21.15.26:0/415332817 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b355aa9660 con 0x55b355b230f0 2024-04-21T10:32:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.458+0000 7fd4daffd640 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55b355b230f0 0x55b355b25510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.459+0000 7fd4daffd640 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55b355b230f0 0x55b355b25510 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa75728bff949383 server_cookie=9747acc2e8222016 in_seq=0 out_seq=0 2024-04-21T10:32:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.459+0000 7fd4baffd640 10 client.15750 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4baffd640 1 -- 172.21.15.26:0/415332817 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b355aa9660 con 0x55b355b230f0 2024-04-21T10:32:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4baffd640 10 client.15750 handle_command_reply: tid=0 2024-04-21T10:32:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4e4574740 1 client.15750 shutdown 2024-04-21T10:32:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4e4574740 1 -- 172.21.15.26:0/415332817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa38d0 msgr2=0x55b355ab4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4e4574740 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa38d0 0x55b355ab4700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd4c409bde0 tx=0x7fd4c40983f0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4e4574740 1 -- 172.21.15.26:0/415332817 shutdown_connections 2024-04-21T10:32:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4e4574740 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55b355b230f0 0x55b355b25510 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4e4574740 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b355aac0e0 0x55b355ab1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4baffd640 0 client.15750 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4e4574740 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b355aa38d0 0x55b355ab4700 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4e4574740 1 --2- 172.21.15.26:0/415332817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b355aa2b60 0x55b355ab41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4e4574740 1 -- 172.21.15.26:0/415332817 >> 172.21.15.26:0/415332817 conn(0x55b355a69c90 msgr2=0x55b355a9c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4e4574740 1 -- 172.21.15.26:0/415332817 shutdown_connections 2024-04-21T10:32:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4e4574740 1 -- 172.21.15.26:0/415332817 wait complete. 2024-04-21T10:32:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.497+0000 7fd4e4574740 20 client.15750 trim_cache size 0 max 16384 2024-04-21T10:32:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-04-21T10:32:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-04-21T10:32:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-04-21T10:32:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-04-21T10:32:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.498+0000 7fd4e1985640 1 -- 172.21.15.26:0/1639786895 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4c80a3ff0 msgr2=0x7fd4c80a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.498+0000 7fd4e1985640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4c80a3ff0 0x7fd4c80a64b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4004af0 tx=0x7fd4d4005c50 comp rx=0 tx=0).stop 2024-04-21T10:32:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.498+0000 7fd4e1985640 1 -- 172.21.15.26:0/1639786895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4dc107290 msgr2=0x7fd4dc198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.498+0000 7fd4e1985640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4dc107290 0x7fd4dc198c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400d740 tx=0x7fd4c400dc00 comp rx=0 tx=0).stop 2024-04-21T10:32:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.498+0000 7fd4e1985640 1 -- 172.21.15.26:0/1639786895 shutdown_connections 2024-04-21T10:32:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.498+0000 7fd4e1985640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4c80a3ff0 0x7fd4c80a64b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.498+0000 7fd4e1985640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4dc19cfb0 0x7fd4dc19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.498+0000 7fd4e1985640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4dc107290 0x7fd4dc198c80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.498+0000 7fd4e1985640 1 --2- 172.21.15.26:0/1639786895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc1021c0 0x7fd4dc198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.498+0000 7fd4e1985640 1 -- 172.21.15.26:0/1639786895 >> 172.21.15.26:0/1639786895 conn(0x7fd4dc0fda20 msgr2=0x7fd4dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.498+0000 7fd4e1985640 1 -- 172.21.15.26:0/1639786895 shutdown_connections 2024-04-21T10:32:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:06.498+0000 7fd4e1985640 1 -- 172.21.15.26:0/1639786895 wait complete. 2024-04-21T10:32:07.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:07 smithi026 ceph-mon[27965]: pgmap v37: 97 pgs: 97 active+clean; 1.5 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 23 op/s 2024-04-21T10:32:07.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:07 smithi169 ceph-mon[31469]: pgmap v37: 97 pgs: 97 active+clean; 1.5 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 23 op/s 2024-04-21T10:32:07.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:07 smithi190 ceph-mon[31423]: pgmap v37: 97 pgs: 97 active+clean; 1.5 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 23 op/s 2024-04-21T10:32:09.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:09 smithi026 ceph-mon[27965]: pgmap v38: 97 pgs: 97 active+clean; 1.5 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 KiB/s wr, 21 op/s 2024-04-21T10:32:09.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:09 smithi026 ceph-mon[27965]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:32:09.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:09 smithi169 ceph-mon[31469]: pgmap v38: 97 pgs: 97 active+clean; 1.5 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 KiB/s wr, 21 op/s 2024-04-21T10:32:09.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:09 smithi169 ceph-mon[31469]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:32:09.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:09 smithi190 ceph-mon[31423]: pgmap v38: 97 pgs: 97 active+clean; 1.5 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 37 KiB/s wr, 21 op/s 2024-04-21T10:32:09.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:09 smithi190 ceph-mon[31423]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:32:11.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:11 smithi026 ceph-mon[27965]: pgmap v39: 97 pgs: 97 active+clean; 15 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.2 MiB/s wr, 42 op/s 2024-04-21T10:32:11.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:11 smithi169 ceph-mon[31469]: pgmap v39: 97 pgs: 97 active+clean; 15 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.2 MiB/s wr, 42 op/s 2024-04-21T10:32:11.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:11 smithi190 ceph-mon[31423]: pgmap v39: 97 pgs: 97 active+clean; 15 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.2 MiB/s wr, 42 op/s 2024-04-21T10:32:13.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:13 smithi026 ceph-mon[27965]: pgmap v40: 97 pgs: 97 active+clean; 15 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.2 MiB/s wr, 39 op/s 2024-04-21T10:32:13.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:13 smithi169 ceph-mon[31469]: pgmap v40: 97 pgs: 97 active+clean; 15 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.2 MiB/s wr, 39 op/s 2024-04-21T10:32:13.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:13 smithi190 ceph-mon[31423]: pgmap v40: 97 pgs: 97 active+clean; 15 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.2 MiB/s wr, 39 op/s 2024-04-21T10:32:14.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:14 smithi026 ceph-mon[27965]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:32:14.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:14 smithi169 ceph-mon[31469]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:32:14.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:14 smithi190 ceph-mon[31423]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:32:15.677 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:15 smithi190 ceph-mon[31423]: pgmap v41: 97 pgs: 97 active+clean; 17 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 47 op/s 2024-04-21T10:32:15.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:15 smithi026 ceph-mon[27965]: pgmap v41: 97 pgs: 97 active+clean; 17 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 47 op/s 2024-04-21T10:32:15.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:15 smithi169 ceph-mon[31469]: pgmap v41: 97 pgs: 97 active+clean; 17 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 47 op/s 2024-04-21T10:32:17.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:17 smithi026 ceph-mon[27965]: pgmap v42: 97 pgs: 97 active+clean; 17 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.3 MiB/s wr, 28 op/s 2024-04-21T10:32:17.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:17 smithi169 ceph-mon[31469]: pgmap v42: 97 pgs: 97 active+clean; 17 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.3 MiB/s wr, 28 op/s 2024-04-21T10:32:17.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:17 smithi190 ceph-mon[31423]: pgmap v42: 97 pgs: 97 active+clean; 17 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.3 MiB/s wr, 28 op/s 2024-04-21T10:32:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.593+0000 7fa27f2c8640 1 -- 172.21.15.26:0/4077955053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa278102b80 msgr2=0x7fa278106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.593+0000 7fa27f2c8640 1 --2- 172.21.15.26:0/4077955053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa278102b80 0x7fa278106bc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009900 tx=0x7fa26c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.594+0000 7fa27f2c8640 1 -- 172.21.15.26:0/4077955053 shutdown_connections 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.594+0000 7fa27f2c8640 1 --2- 172.21.15.26:0/4077955053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2781072a0 0x7fa27810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.594+0000 7fa27f2c8640 1 --2- 172.21.15.26:0/4077955053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa278102b80 0x7fa278106bc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.594+0000 7fa27f2c8640 1 --2- 172.21.15.26:0/4077955053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2781021d0 0x7fa2781025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.594+0000 7fa27f2c8640 1 -- 172.21.15.26:0/4077955053 >> 172.21.15.26:0/4077955053 conn(0x7fa2780793e0 msgr2=0x7fa2780797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.594+0000 7fa27f2c8640 1 -- 172.21.15.26:0/4077955053 shutdown_connections 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.594+0000 7fa27f2c8640 1 -- 172.21.15.26:0/4077955053 wait complete. 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.594+0000 7fa27f2c8640 1 Processor -- start 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.594+0000 7fa27f2c8640 1 -- start start 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.594+0000 7fa27f2c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2781021d0 0x7fa2781986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27f2c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2781072a0 0x7fa278198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27f2c8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa27819cf40 0x7fa27819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27f2c8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa278112230 con 0x7fa2781072a0 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27f2c8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781120b0 con 0x7fa27819cf40 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27f2c8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781123b0 con 0x7fa2781021d0 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27c863640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2781072a0 0x7fa278198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27c863640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2781072a0 0x7fa278198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43694/0 (socket says 172.21.15.26:43694) 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27c863640 1 -- 172.21.15.26:0/826456877 learned_addr learned my addr 172.21.15.26:0/826456877 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27d064640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2781021d0 0x7fa2781986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27d865640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa27819cf40 0x7fa27819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27c863640 1 -- 172.21.15.26:0/826456877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2781021d0 msgr2=0x7fa2781986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27c863640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2781021d0 0x7fa2781986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27c863640 1 -- 172.21.15.26:0/826456877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa27819cf40 msgr2=0x7fa27819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27c863640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa27819cf40 0x7fa27819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27c863640 1 -- 172.21.15.26:0/826456877 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27819da30 con 0x7fa2781072a0 2024-04-21T10:32:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27d865640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa27819cf40 0x7fa27819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:32:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27c863640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2781072a0 0x7fa278198c10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa26c000c00 tx=0x7fa26c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa26a7fc640 1 -- 172.21.15.26:0/826456877 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c043020 con 0x7fa2781072a0 2024-04-21T10:32:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa26a7fc640 1 -- 172.21.15.26:0/826456877 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa26c02cb70 con 0x7fa2781072a0 2024-04-21T10:32:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27f2c8640 1 -- 172.21.15.26:0/826456877 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27819dcc0 con 0x7fa2781072a0 2024-04-21T10:32:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa26a7fc640 1 -- 172.21.15.26:0/826456877 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c003b80 con 0x7fa2781072a0 2024-04-21T10:32:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.595+0000 7fa27f2c8640 1 -- 172.21.15.26:0/826456877 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2781a54e0 con 0x7fa2781072a0 2024-04-21T10:32:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa281f99740 1 Processor -- start 2024-04-21T10:32:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa281f99740 1 -- start start 2024-04-21T10:32:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa26a7fc640 1 -- 172.21.15.26:0/826456877 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fa26c046020 con 0x7fa2781072a0 2024-04-21T10:32:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa281f99740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 0x55cfe59e5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa281f99740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e5b60 0x55cfe59e5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa281f99740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59ef0e0 0x55cfe59f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa281f99740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cfe594a8d0 con 0x55cfe59e68d0 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa281f99740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cfe58e7e80 con 0x55cfe59e5b60 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa281f99740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cfe5946890 con 0x55cfe59ef0e0 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa27d865640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 0x55cfe59e5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa27d865640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 0x55cfe59e5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43708/0 (socket says 172.21.15.26:43708) 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa27d865640 1 -- 172.21.15.26:0/3991864515 learned_addr learned my addr 172.21.15.26:0/3991864515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa27c863640 1 --2- 172.21.15.26:0/3991864515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59ef0e0 0x55cfe59f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.598+0000 7fa27d064640 1 --2- 172.21.15.26:0/3991864515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e5b60 0x55cfe59e5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa27d865640 1 -- 172.21.15.26:0/3991864515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59ef0e0 msgr2=0x55cfe59f14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa27d865640 1 --2- 172.21.15.26:0/3991864515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59ef0e0 0x55cfe59f14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa27d865640 1 -- 172.21.15.26:0/3991864515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e5b60 msgr2=0x55cfe59e5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa27d865640 1 --2- 172.21.15.26:0/3991864515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e5b60 0x55cfe59e5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa27d865640 1 -- 172.21.15.26:0/3991864515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe5909320 con 0x55cfe59e68d0 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa27d064640 1 --2- 172.21.15.26:0/3991864515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e5b60 0x55cfe59e5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa27d865640 1 --2- 172.21.15.26:0/3991864515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 0x55cfe59e5620 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa270009b10 tx=0x7fa270010370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc107573c8df1da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa26a7fc640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fa2640a3f70 0x7fa2640a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa2637fe640 1 -- 172.21.15.26:0/3991864515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270010d70 con 0x55cfe59e68d0 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa2637fe640 1 -- 172.21.15.26:0/3991864515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2700035f0 con 0x55cfe59e68d0 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa2637fe640 1 -- 172.21.15.26:0/3991864515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2700044e0 con 0x55cfe59e68d0 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa27d064640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fa2640a3f70 0x7fa2640a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.599+0000 7fa26a7fc640 1 -- 172.21.15.26:0/826456877 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fa26c0f2b00 con 0x7fa2781072a0 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.600+0000 7fa281f99740 1 -- 172.21.15.26:0/3991864515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 msgr2=0x55cfe59e5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.600+0000 7fa27d064640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fa2640a3f70 0x7fa2640a6430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55cfe59e7030 tx=0x7fa274009680 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.600+0000 7fa281f99740 1 --2- 172.21.15.26:0/3991864515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 0x55cfe59e5620 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa270009b10 tx=0x7fa270010370 comp rx=0 tx=0).stop 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.600+0000 7fa281f99740 1 -- 172.21.15.26:0/3991864515 shutdown_connections 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.600+0000 7fa281f99740 1 --2- 172.21.15.26:0/3991864515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59ef0e0 0x55cfe59f14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.600+0000 7fa281f99740 1 --2- 172.21.15.26:0/3991864515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e5b60 0x55cfe59e5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.600+0000 7fa281f99740 1 --2- 172.21.15.26:0/3991864515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 0x55cfe59e5620 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.600+0000 7fa281f99740 1 -- 172.21.15.26:0/3991864515 >> 172.21.15.26:0/3991864515 conn(0x55cfe59acc90 msgr2=0x55cfe59e0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.600+0000 7fa281f99740 1 -- 172.21.15.26:0/3991864515 shutdown_connections 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.600+0000 7fa281f99740 1 -- 172.21.15.26:0/3991864515 wait complete. 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.600+0000 7fa281f99740 1 -- start start 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.600+0000 7fa281f99740 1 Processor -- start 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa281f99740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 0x55cfe59eea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa281f99740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 0x55cfe59ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa281f99740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59ef0e0 0x55cfe59ec380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa281f99740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cfe59f1e10 con 0x55cfe59e68d0 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa281f99740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cfe58d2eb0 con 0x55cfe59ef0e0 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa281f99740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cfe59f1f90 con 0x55cfe59e5b60 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa281f99740 10 client.0 _ll_register_callbacks cb 0x55cfe58ffb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa27c863640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 0x55cfe59ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa27c863640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 0x55cfe59ebe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43714/0 (socket says 172.21.15.26:43714) 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa27c863640 1 -- 172.21.15.26:0/2538147501 learned_addr learned my addr 172.21.15.26:0/2538147501 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa27d064640 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 0x55cfe59eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa27d865640 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59ef0e0 0x55cfe59ec380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa27c863640 1 -- 172.21.15.26:0/2538147501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 msgr2=0x55cfe59eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa27c863640 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 0x55cfe59eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa27c863640 1 -- 172.21.15.26:0/2538147501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59ef0e0 msgr2=0x55cfe59ec380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa27c863640 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59ef0e0 0x55cfe59ec380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa27c863640 1 -- 172.21.15.26:0/2538147501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe565d210 con 0x55cfe59e68d0 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa27d865640 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59ef0e0 0x55cfe59ec380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa27c863640 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 0x55cfe59ebe00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa26c006010 tx=0x7fa26c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b82ecf2931a4c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa260ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa260ff9640 1 -- 172.21.15.26:0/2538147501 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0f8030 con 0x55cfe59e68d0 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa260ff9640 1 -- 172.21.15.26:0/2538147501 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fa26c005490 con 0x55cfe59e68d0 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa281f99740 1 -- 172.21.15.26:0/2538147501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2700035f0 con 0x55cfe59e68d0 2024-04-21T10:32:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.601+0000 7fa260ff9640 1 -- 172.21.15.26:0/2538147501 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa26c0be4c0 con 0x55cfe59e68d0 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.602+0000 7fa281f99740 10 client.15762 fetch_fsmap learned FSMap version 24 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.602+0000 7fa281f99740 1 -- 172.21.15.26:0/2538147501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cfe5729620 con 0x55cfe59e68d0 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.602+0000 7fa260ff9640 1 -- 172.21.15.26:0/2538147501 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa26c0da070 con 0x55cfe59e68d0 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.602+0000 7fa260ff9640 1 -- 172.21.15.26:0/2538147501 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fa26c0c7160 con 0x55cfe59e68d0 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.602+0000 7fa281f99740 10 client.15762 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.602+0000 7fa281f99740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.602+0000 7fa281f99740 20 client.15762 populate_metadata read hostname 'smithi026' 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.602+0000 7fa281f99740 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55cfe59ff120 0x55cfe5a01540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.602+0000 7fa281f99740 4 client.15762 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.602+0000 7fa281f99740 1 -- 172.21.15.26:0/2538147501 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfe595e880 con 0x55cfe59ff120 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.602+0000 7fa27d064640 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55cfe59ff120 0x55cfe5a01540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.603+0000 7fa27d064640 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55cfe59ff120 0x55cfe5a01540 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a841ff9968e228a7 server_cookie=480b2897890092ca in_seq=0 out_seq=0 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.603+0000 7fa260ff9640 10 client.15762 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa260ff9640 1 -- 172.21.15.26:0/2538147501 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfe595e880 con 0x55cfe59ff120 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa260ff9640 10 client.15762 handle_command_reply: tid=0 2024-04-21T10:32:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa281f99740 1 client.15762 shutdown 2024-04-21T10:32:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa281f99740 1 -- 172.21.15.26:0/2538147501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 msgr2=0x55cfe59ebe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa281f99740 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 0x55cfe59ebe00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa26c006010 tx=0x7fa26c0035f0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa281f99740 1 -- 172.21.15.26:0/2538147501 shutdown_connections 2024-04-21T10:32:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa281f99740 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55cfe59ff120 0x55cfe5a01540 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa281f99740 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59ef0e0 0x55cfe59ec380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa281f99740 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59e68d0 0x55cfe59ebe00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa281f99740 1 --2- 172.21.15.26:0/2538147501 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 0x55cfe59eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa260ff9640 0 client.15762 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa281f99740 1 -- 172.21.15.26:0/2538147501 >> 172.21.15.26:0/2538147501 conn(0x55cfe59acc90 msgr2=0x55cfe59e25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa281f99740 1 -- 172.21.15.26:0/2538147501 shutdown_connections 2024-04-21T10:32:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa281f99740 1 -- 172.21.15.26:0/2538147501 wait complete. 2024-04-21T10:32:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.604+0000 7fa281f99740 20 client.15762 trim_cache size 0 max 16384 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.619+0000 7fa281f99740 1 Processor -- start 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.619+0000 7fa281f99740 1 -- start start 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.619+0000 7fa281f99740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 0x55cfe59fad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.619+0000 7fa281f99740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e68d0 0x55cfe59ee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.619+0000 7fa281f99740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 0x55cfe59eea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.619+0000 7fa281f99740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781b03e0 con 0x55cfe59ef0e0 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.619+0000 7fa281f99740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781adc50 con 0x55cfe59e68d0 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.619+0000 7fa281f99740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781abac0 con 0x55cfe59e5b60 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.619+0000 7fa27d865640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 0x55cfe59eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.619+0000 7fa27d865640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 0x55cfe59eea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43716/0 (socket says 172.21.15.26:43716) 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.619+0000 7fa27d865640 1 -- 172.21.15.26:0/1536675305 learned_addr learned my addr 172.21.15.26:0/1536675305 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.620+0000 7fa27d064640 1 --2- 172.21.15.26:0/1536675305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 0x55cfe59fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.620+0000 7fa27c863640 1 --2- 172.21.15.26:0/1536675305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e68d0 0x55cfe59ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.620+0000 7fa27d865640 1 -- 172.21.15.26:0/1536675305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 msgr2=0x55cfe59fad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.620+0000 7fa27d865640 1 --2- 172.21.15.26:0/1536675305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 0x55cfe59fad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.620+0000 7fa27d865640 1 -- 172.21.15.26:0/1536675305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e68d0 msgr2=0x55cfe59ee530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.620+0000 7fa27d865640 1 --2- 172.21.15.26:0/1536675305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e68d0 0x55cfe59ee530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.620+0000 7fa27d865640 1 -- 172.21.15.26:0/1536675305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe59e4e30 con 0x55cfe59ef0e0 2024-04-21T10:32:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.620+0000 7fa27d865640 1 --2- 172.21.15.26:0/1536675305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 0x55cfe59eea70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098e0 tx=0x7fa270005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae1a1a87af2b3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.620+0000 7fa262ffd640 1 -- 172.21.15.26:0/1536675305 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27002a070 con 0x55cfe59ef0e0 2024-04-21T10:32:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.620+0000 7fa262ffd640 1 -- 172.21.15.26:0/1536675305 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa270005570 con 0x55cfe59ef0e0 2024-04-21T10:32:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.620+0000 7fa262ffd640 1 -- 172.21.15.26:0/1536675305 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270012450 con 0x55cfe59ef0e0 2024-04-21T10:32:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.621+0000 7fa281f99740 1 -- 172.21.15.26:0/1536675305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 msgr2=0x55cfe59eea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.621+0000 7fa281f99740 1 --2- 172.21.15.26:0/1536675305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 0x55cfe59eea70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098e0 tx=0x7fa270005d30 comp rx=0 tx=0).stop 2024-04-21T10:32:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.621+0000 7fa281f99740 1 -- 172.21.15.26:0/1536675305 shutdown_connections 2024-04-21T10:32:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.621+0000 7fa281f99740 1 --2- 172.21.15.26:0/1536675305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 0x55cfe59eea70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.621+0000 7fa281f99740 1 --2- 172.21.15.26:0/1536675305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e68d0 0x55cfe59ee530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.621+0000 7fa281f99740 1 --2- 172.21.15.26:0/1536675305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 0x55cfe59fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.621+0000 7fa281f99740 1 -- 172.21.15.26:0/1536675305 >> 172.21.15.26:0/1536675305 conn(0x55cfe59acc90 msgr2=0x55cfe59e0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.621+0000 7fa281f99740 1 -- 172.21.15.26:0/1536675305 shutdown_connections 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.621+0000 7fa281f99740 1 -- 172.21.15.26:0/1536675305 wait complete. 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.621+0000 7fa281f99740 1 -- start start 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.621+0000 7fa281f99740 1 Processor -- start 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa281f99740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 0x55cfe59f7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa281f99740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e68d0 0x55cfe59f7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa281f99740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 0x55cfe59e8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa281f99740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781ac110 con 0x55cfe59ef0e0 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa281f99740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781ad580 con 0x55cfe59e68d0 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa281f99740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781ac7e0 con 0x55cfe59e5b60 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa281f99740 10 client.0 _ll_register_callbacks cb 0x7fa26c009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:32:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa27d865640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 0x55cfe59e8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa27d865640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 0x55cfe59e8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43720/0 (socket says 172.21.15.26:43720) 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa27d865640 1 -- 172.21.15.26:0/3028046487 learned_addr learned my addr 172.21.15.26:0/3028046487 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa27d064640 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 0x55cfe59f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa27c863640 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e68d0 0x55cfe59f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa27d865640 1 -- 172.21.15.26:0/3028046487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 msgr2=0x55cfe59f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa27d865640 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 0x55cfe59f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa27d865640 1 -- 172.21.15.26:0/3028046487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e68d0 msgr2=0x55cfe59f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa27d865640 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e68d0 0x55cfe59f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa27d865640 1 -- 172.21.15.26:0/3028046487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe59e8e50 con 0x55cfe59ef0e0 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa27d865640 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 0x55cfe59e8650 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa2700103b0 tx=0x7fa2700039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce2600e1098439 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa260ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa260ff9640 1 -- 172.21.15.26:0/3028046487 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27002a070 con 0x55cfe59ef0e0 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa260ff9640 1 -- 172.21.15.26:0/3028046487 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fa270003ba0 con 0x55cfe59ef0e0 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa281f99740 1 -- 172.21.15.26:0/3028046487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa270005570 con 0x55cfe59ef0e0 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.622+0000 7fa260ff9640 1 -- 172.21.15.26:0/3028046487 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa270003da0 con 0x55cfe59ef0e0 2024-04-21T10:32:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa281f99740 10 client.15768 fetch_fsmap learned FSMap version 24 2024-04-21T10:32:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa281f99740 1 -- 172.21.15.26:0/3028046487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cfe5a65250 con 0x55cfe59ef0e0 2024-04-21T10:32:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa260ff9640 1 -- 172.21.15.26:0/3028046487 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa270005570 con 0x55cfe59ef0e0 2024-04-21T10:32:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa260ff9640 1 -- 172.21.15.26:0/3028046487 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fa27000f040 con 0x55cfe59ef0e0 2024-04-21T10:32:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa281f99740 10 client.15768 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T10:32:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa281f99740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:32:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa281f99740 20 client.15768 populate_metadata read hostname 'smithi026' 2024-04-21T10:32:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa281f99740 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55cfe5a65770 0x55cfe5a67b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa281f99740 4 client.15768 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T10:32:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa281f99740 1 -- 172.21.15.26:0/3028046487 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cfe58ea3d0 con 0x55cfe5a65770 2024-04-21T10:32:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa27d064640 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55cfe5a65770 0x55cfe5a67b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa260ff9640 10 client.15768 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.623+0000 7fa27d064640 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55cfe5a65770 0x55cfe5a67b90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47195d2407ffa7ef server_cookie=dce6cbc77501713f in_seq=0 out_seq=0 2024-04-21T10:32:19.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:19 smithi026 ceph-mon[27965]: pgmap v43: 97 pgs: 97 active+clean; 17 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.3 MiB/s wr, 28 op/s 2024-04-21T10:32:19.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:19 smithi169 ceph-mon[31469]: pgmap v43: 97 pgs: 97 active+clean; 17 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.3 MiB/s wr, 28 op/s 2024-04-21T10:32:19.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:19 smithi190 ceph-mon[31423]: pgmap v43: 97 pgs: 97 active+clean; 17 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.3 MiB/s wr, 28 op/s 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.872+0000 7fa260ff9640 1 -- 172.21.15.26:0/3028046487 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cfe58ea3d0 con 0x55cfe5a65770 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.872+0000 7fa260ff9640 10 client.15768 handle_command_reply: tid=0 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.872+0000 7fa281f99740 1 client.15768 shutdown 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa281f99740 1 -- 172.21.15.26:0/3028046487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 msgr2=0x55cfe59e8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa281f99740 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 0x55cfe59e8650 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa2700103b0 tx=0x7fa2700039d0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa281f99740 1 -- 172.21.15.26:0/3028046487 shutdown_connections 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa281f99740 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55cfe5a65770 0x55cfe5a67b90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa281f99740 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe59ef0e0 0x55cfe59e8650 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa281f99740 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cfe59e68d0 0x55cfe59f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa281f99740 1 --2- 172.21.15.26:0/3028046487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cfe59e5b60 0x55cfe59f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa260ff9640 0 client.15768 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa281f99740 1 -- 172.21.15.26:0/3028046487 >> 172.21.15.26:0/3028046487 conn(0x55cfe59acc90 msgr2=0x55cfe59ea610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa281f99740 1 -- 172.21.15.26:0/3028046487 shutdown_connections 2024-04-21T10:32:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa281f99740 1 -- 172.21.15.26:0/3028046487 wait complete. 2024-04-21T10:32:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa281f99740 20 client.15768 trim_cache size 0 max 16384 2024-04-21T10:32:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-04-21T10:32:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-04-21T10:32:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-04-21T10:32:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-04-21T10:32:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.873+0000 7fa27f2c8640 1 -- 172.21.15.26:0/826456877 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fa2640a3f70 msgr2=0x7fa2640a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.874+0000 7fa27f2c8640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fa2640a3f70 0x7fa2640a6430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55cfe59e7030 tx=0x7fa274009680 comp rx=0 tx=0).stop 2024-04-21T10:32:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.874+0000 7fa27f2c8640 1 -- 172.21.15.26:0/826456877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2781072a0 msgr2=0x7fa278198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.874+0000 7fa27f2c8640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2781072a0 0x7fa278198c10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa26c000c00 tx=0x7fa26c0045e0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.874+0000 7fa27f2c8640 1 -- 172.21.15.26:0/826456877 shutdown_connections 2024-04-21T10:32:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.874+0000 7fa27f2c8640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fa2640a3f70 0x7fa2640a6430 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.874+0000 7fa27f2c8640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa27819cf40 0x7fa27819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.874+0000 7fa27f2c8640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2781072a0 0x7fa278198c10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.874+0000 7fa27f2c8640 1 --2- 172.21.15.26:0/826456877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2781021d0 0x7fa2781986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.874+0000 7fa27f2c8640 1 -- 172.21.15.26:0/826456877 >> 172.21.15.26:0/826456877 conn(0x7fa2780793e0 msgr2=0x7fa2780ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.874+0000 7fa27f2c8640 1 -- 172.21.15.26:0/826456877 shutdown_connections 2024-04-21T10:32:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:19.874+0000 7fa27f2c8640 1 -- 172.21.15.26:0/826456877 wait complete. 2024-04-21T10:32:20.404 INFO:tasks.workunit.client.0.smithi026.stderr:Updating files: 11% (491/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 13% (575/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 20% (858/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 46% (1974/4290) Updating files: 46% (1983/4290) Updating files: 47% (2017/4290) Updating files: 48% (2060/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 60% (2590/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 76% (3295/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 86% (3690/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 92% (3979/4290) Updating files: 93% (3990/4290) Updating files: 94% (4033/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-21T10:32:20.413 INFO:tasks.workunit.client.0.smithi026.stderr:+ cd xfstests-dev 2024-04-21T10:32:20.413 INFO:tasks.workunit.client.0.smithi026.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-04-21T10:32:21.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:21 smithi026 ceph-mon[27965]: pgmap v44: 97 pgs: 97 active+clean; 23 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.9 MiB/s wr, 63 op/s 2024-04-21T10:32:21.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:21 smithi169 ceph-mon[31469]: pgmap v44: 97 pgs: 97 active+clean; 23 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.9 MiB/s wr, 63 op/s 2024-04-21T10:32:21.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:21 smithi190 ceph-mon[31423]: pgmap v44: 97 pgs: 97 active+clean; 23 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.9 MiB/s wr, 63 op/s 2024-04-21T10:32:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:23 smithi026 ceph-mon[27965]: pgmap v45: 97 pgs: 97 active+clean; 23 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 764 KiB/s wr, 42 op/s 2024-04-21T10:32:23.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:23 smithi169 ceph-mon[31469]: pgmap v45: 97 pgs: 97 active+clean; 23 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 764 KiB/s wr, 42 op/s 2024-04-21T10:32:23.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:23 smithi190 ceph-mon[31423]: pgmap v45: 97 pgs: 97 active+clean; 23 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 764 KiB/s wr, 42 op/s 2024-04-21T10:32:25.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:25 smithi026 ceph-mon[27965]: pgmap v46: 97 pgs: 97 active+clean; 23 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 764 KiB/s wr, 42 op/s 2024-04-21T10:32:25.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:25 smithi169 ceph-mon[31469]: pgmap v46: 97 pgs: 97 active+clean; 23 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 764 KiB/s wr, 42 op/s 2024-04-21T10:32:25.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:25 smithi190 ceph-mon[31423]: pgmap v46: 97 pgs: 97 active+clean; 23 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 764 KiB/s wr, 42 op/s 2024-04-21T10:32:27.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:27 smithi026 ceph-mon[27965]: pgmap v47: 97 pgs: 97 active+clean; 35 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 493 op/s 2024-04-21T10:32:27.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:27 smithi169 ceph-mon[31469]: pgmap v47: 97 pgs: 97 active+clean; 35 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 493 op/s 2024-04-21T10:32:27.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:27 smithi190 ceph-mon[31423]: pgmap v47: 97 pgs: 97 active+clean; 35 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 493 op/s 2024-04-21T10:32:29.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:29 smithi026 ceph-mon[27965]: pgmap v48: 97 pgs: 97 active+clean; 35 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 493 op/s 2024-04-21T10:32:29.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:29 smithi169 ceph-mon[31469]: pgmap v48: 97 pgs: 97 active+clean; 35 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 493 op/s 2024-04-21T10:32:29.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:29 smithi190 ceph-mon[31423]: pgmap v48: 97 pgs: 97 active+clean; 35 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 493 op/s 2024-04-21T10:32:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:31 smithi026 ceph-mon[27965]: pgmap v49: 97 pgs: 97 active+clean; 48 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 789 op/s 2024-04-21T10:32:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:31 smithi026 ceph-mon[27965]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-04-21T10:32:31.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:31 smithi169 ceph-mon[31469]: pgmap v49: 97 pgs: 97 active+clean; 48 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 789 op/s 2024-04-21T10:32:31.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:31 smithi169 ceph-mon[31469]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-04-21T10:32:31.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:31 smithi190 ceph-mon[31423]: pgmap v49: 97 pgs: 97 active+clean; 48 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 789 op/s 2024-04-21T10:32:31.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:31 smithi190 ceph-mon[31423]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-04-21T10:32:32.200 INFO:tasks.workunit.client.0.smithi026.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-04-21T10:32:32.200 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:32:32.200 INFO:tasks.workunit.client.0.smithi026.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-21T10:32:32.200 INFO:tasks.workunit.client.0.smithi026.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-21T10:32:32.200 INFO:tasks.workunit.client.0.smithi026.stderr:state without impacting any branches by switching back to a branch. 2024-04-21T10:32:32.200 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:32:32.200 INFO:tasks.workunit.client.0.smithi026.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-21T10:32:32.200 INFO:tasks.workunit.client.0.smithi026.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-21T10:32:32.200 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:32:32.201 INFO:tasks.workunit.client.0.smithi026.stderr: git switch -c 2024-04-21T10:32:32.201 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:32:32.201 INFO:tasks.workunit.client.0.smithi026.stderr:Or undo this operation with: 2024-04-21T10:32:32.201 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:32:32.201 INFO:tasks.workunit.client.0.smithi026.stderr: git switch - 2024-04-21T10:32:32.201 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:32:32.201 INFO:tasks.workunit.client.0.smithi026.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-21T10:32:32.201 INFO:tasks.workunit.client.0.smithi026.stderr: 2024-04-21T10:32:32.201 INFO:tasks.workunit.client.0.smithi026.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-04-21T10:32:32.206 INFO:tasks.workunit.client.0.smithi026.stderr:+ make -j4 2024-04-21T10:32:32.214 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize -cfi 2024-04-21T10:32:32.565 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-21T10:32:32.566 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize: copying file './config.guess' 2024-04-21T10:32:32.570 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize: copying file './config.sub' 2024-04-21T10:32:32.575 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize: copying file './install-sh' 2024-04-21T10:32:32.580 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize: copying file './ltmain.sh' 2024-04-21T10:32:32.603 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-21T10:32:32.603 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-21T10:32:32.609 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-21T10:32:32.614 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-21T10:32:32.620 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-21T10:32:32.626 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-21T10:32:32.632 INFO:tasks.workunit.client.0.smithi026.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-21T10:32:32.633 INFO:tasks.workunit.client.0.smithi026.stdout:cp include/install-sh . 2024-04-21T10:32:32.642 INFO:tasks.workunit.client.0.smithi026.stdout:aclocal -I m4 2024-04-21T10:32:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.964+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2648510291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d14106190 msgr2=0x7f7d1410dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.964+0000 7f7d1c6a5640 1 --2- 172.21.15.26:0/2648510291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d14106190 0x7f7d1410dd60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009ca0 tx=0x7f7d0c02c0f0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.964+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2648510291 shutdown_connections 2024-04-21T10:32:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.964+0000 7f7d1c6a5640 1 --2- 172.21.15.26:0/2648510291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d14106190 0x7f7d1410dd60 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.964+0000 7f7d1c6a5640 1 --2- 172.21.15.26:0/2648510291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d14101a70 0x7f7d14105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.964+0000 7f7d1c6a5640 1 --2- 172.21.15.26:0/2648510291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d141010c0 0x7f7d141014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.964+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2648510291 >> 172.21.15.26:0/2648510291 conn(0x7f7d14077f60 msgr2=0x7f7d14078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.964+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2648510291 shutdown_connections 2024-04-21T10:32:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.964+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2648510291 wait complete. 2024-04-21T10:32:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.964+0000 7f7d1c6a5640 1 Processor -- start 2024-04-21T10:32:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d1c6a5640 1 -- start start 2024-04-21T10:32:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d1c6a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d141010c0 0x7f7d141986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d1c6a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d14106190 0x7f7d14198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d1c6a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d1419cf70 0x7f7d1419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d1c6a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14111100 con 0x7f7d14106190 2024-04-21T10:32:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d1c6a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14110f80 con 0x7f7d1419cf70 2024-04-21T10:32:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d1c6a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14111280 con 0x7f7d141010c0 2024-04-21T10:32:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d19c40640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d14106190 0x7f7d14198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d19c40640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d14106190 0x7f7d14198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39846/0 (socket says 172.21.15.26:39846) 2024-04-21T10:32:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d19c40640 1 -- 172.21.15.26:0/2715277745 learned_addr learned my addr 172.21.15.26:0/2715277745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d1a441640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d141010c0 0x7f7d141986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d1419cf70 0x7f7d1419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d19c40640 1 -- 172.21.15.26:0/2715277745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d141010c0 msgr2=0x7f7d141986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d19c40640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d141010c0 0x7f7d141986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d19c40640 1 -- 172.21.15.26:0/2715277745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d1419cf70 msgr2=0x7f7d1419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d19c40640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d1419cf70 0x7f7d1419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d19c40640 1 -- 172.21.15.26:0/2715277745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1419da60 con 0x7f7d14106190 2024-04-21T10:32:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d1419cf70 0x7f7d1419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:32:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.965+0000 7f7d19c40640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d14106190 0x7f7d14198bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800e9c0 tx=0x7f7d0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.966+0000 7f7d077fe640 1 -- 172.21.15.26:0/2715277745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0800cd80 con 0x7f7d14106190 2024-04-21T10:32:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.966+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2715277745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1419dd50 con 0x7f7d14106190 2024-04-21T10:32:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.966+0000 7f7d077fe640 1 -- 172.21.15.26:0/2715277745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d080054d0 con 0x7f7d14106190 2024-04-21T10:32:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.966+0000 7f7d077fe640 1 -- 172.21.15.26:0/2715277745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08010430 con 0x7f7d14106190 2024-04-21T10:32:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.966+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2715277745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d141a5540 con 0x7f7d14106190 2024-04-21T10:32:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.968+0000 7f7d077fe640 1 -- 172.21.15.26:0/2715277745 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f7d08004fd0 con 0x7f7d14106190 2024-04-21T10:32:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.968+0000 7f7d1f371740 1 Processor -- start 2024-04-21T10:32:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.968+0000 7f7d1f371740 1 -- start start 2024-04-21T10:32:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1f371740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c0f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1f371740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c0fb60 0x55daf3c0ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1f371740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c1b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1f371740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55daf3b748d0 con 0x55daf3c108d0 2024-04-21T10:32:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1f371740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55daf3b11e80 con 0x55daf3c190e0 2024-04-21T10:32:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1f371740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55daf3b70890 con 0x55daf3c0fb60 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1ac42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c0f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1ac42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c0f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39854/0 (socket says 172.21.15.26:39854) 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1ac42640 1 -- 172.21.15.26:0/1265186499 learned_addr learned my addr 172.21.15.26:0/1265186499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1a441640 1 --2- 172.21.15.26:0/1265186499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c0fb60 0x55daf3c0ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d19c40640 1 --2- 172.21.15.26:0/1265186499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c1b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1a441640 1 -- 172.21.15.26:0/1265186499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 msgr2=0x55daf3c1b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1a441640 1 --2- 172.21.15.26:0/1265186499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c1b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1a441640 1 -- 172.21.15.26:0/1265186499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 msgr2=0x55daf3c0f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1a441640 1 --2- 172.21.15.26:0/1265186499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c0f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d077fe640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f7d000a3dc0 0x7f7d000a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1a441640 1 -- 172.21.15.26:0/1265186499 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf3b33320 con 0x55daf3c0fb60 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/1265186499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c0f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d19c40640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f7d000a3dc0 0x7f7d000a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d1a441640 1 --2- 172.21.15.26:0/1265186499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c0fb60 0x55daf3c0ffe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009c00 tx=0x7f7d10011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7e52b526d6c872b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.969+0000 7f7d077fe640 1 -- 172.21.15.26:0/2715277745 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f7d080d0c10 con 0x7f7d14106190 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.970+0000 7f7d04ff9640 1 -- 172.21.15.26:0/1265186499 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10004230 con 0x55daf3c0fb60 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.970+0000 7f7d04ff9640 1 -- 172.21.15.26:0/1265186499 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d100043d0 con 0x55daf3c0fb60 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.970+0000 7f7d04ff9640 1 -- 172.21.15.26:0/1265186499 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10004c30 con 0x55daf3c0fb60 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.970+0000 7f7d19c40640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f7d000a3dc0 0x7f7d000a6280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d080ce420 tx=0x7f7d080049b0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.970+0000 7f7d1f371740 1 -- 172.21.15.26:0/1265186499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c0fb60 msgr2=0x55daf3c0ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.970+0000 7f7d1f371740 1 --2- 172.21.15.26:0/1265186499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c0fb60 0x55daf3c0ffe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009c00 tx=0x7f7d10011810 comp rx=0 tx=0).stop 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 -- 172.21.15.26:0/1265186499 shutdown_connections 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 --2- 172.21.15.26:0/1265186499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c1b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 --2- 172.21.15.26:0/1265186499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c0fb60 0x55daf3c0ffe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 --2- 172.21.15.26:0/1265186499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c0f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 -- 172.21.15.26:0/1265186499 >> 172.21.15.26:0/1265186499 conn(0x55daf3bd6c90 msgr2=0x55daf3c0a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 -- 172.21.15.26:0/1265186499 shutdown_connections 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 -- 172.21.15.26:0/1265186499 wait complete. 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 -- start start 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 Processor -- start 2024-04-21T10:32:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c0fb60 0x55daf3c18a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c108d0 0x55daf3c15db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55daf3c1be10 con 0x55daf3c0fb60 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55daf3afceb0 con 0x55daf3c190e0 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55daf3c1bf90 con 0x55daf3c108d0 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.971+0000 7f7d1f371740 10 client.0 _ll_register_callbacks cb 0x55daf3b29b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1ac42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c0fb60 0x55daf3c18a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1ac42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c0fb60 0x55daf3c18a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39862/0 (socket says 172.21.15.26:39862) 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1ac42640 1 -- 172.21.15.26:0/3131236820 learned_addr learned my addr 172.21.15.26:0/3131236820 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1a441640 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c108d0 0x55daf3c15db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1ac42640 1 -- 172.21.15.26:0/3131236820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c108d0 msgr2=0x55daf3c15db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c108d0 0x55daf3c15db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1ac42640 1 -- 172.21.15.26:0/3131236820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 msgr2=0x55daf3c162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1ac42640 1 -- 172.21.15.26:0/3131236820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf3887210 con 0x55daf3c0fb60 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c0fb60 0x55daf3c18a20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c005e30 tx=0x7f7d0c004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9aeb84354efcf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7cfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7cfdffb640 1 -- 172.21.15.26:0/3131236820 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c039040 con 0x55daf3c0fb60 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1f371740 1 -- 172.21.15.26:0/3131236820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d100043d0 con 0x55daf3c0fb60 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7cfdffb640 1 -- 172.21.15.26:0/3131236820 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f7d0c003b70 con 0x55daf3c0fb60 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7cfdffb640 1 -- 172.21.15.26:0/3131236820 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7d0c003d70 con 0x55daf3c0fb60 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1f371740 10 client.15780 fetch_fsmap learned FSMap version 24 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.972+0000 7f7d1f371740 1 -- 172.21.15.26:0/3131236820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55daf3953620 con 0x55daf3c0fb60 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.973+0000 7f7cfdffb640 1 -- 172.21.15.26:0/3131236820 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d100043d0 con 0x55daf3c0fb60 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.973+0000 7f7cfdffb640 1 -- 172.21.15.26:0/3131236820 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f7d0c04d5e0 con 0x55daf3c0fb60 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.973+0000 7f7d1f371740 10 client.15780 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.973+0000 7f7d1f371740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:32:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.973+0000 7f7d1f371740 20 client.15780 populate_metadata read hostname 'smithi026' 2024-04-21T10:32:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.973+0000 7f7d1f371740 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55daf3c29090 0x55daf3c2b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.973+0000 7f7d1f371740 4 client.15780 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:32:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.973+0000 7f7d1f371740 1 -- 172.21.15.26:0/3131236820 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daf3b88880 con 0x55daf3c29090 2024-04-21T10:32:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.974+0000 7f7d1a441640 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55daf3c29090 0x55daf3c2b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.974+0000 7f7d1a441640 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55daf3c29090 0x55daf3c2b4e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b43c7f198e898ff0 server_cookie=16a9bb7667fe2ca7 in_seq=0 out_seq=0 2024-04-21T10:32:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.974+0000 7f7cfdffb640 10 client.15780 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7cfdffb640 1 -- 172.21.15.26:0/3131236820 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55daf3b88880 con 0x55daf3c29090 2024-04-21T10:32:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7cfdffb640 10 client.15780 handle_command_reply: tid=0 2024-04-21T10:32:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7d1f371740 1 client.15780 shutdown 2024-04-21T10:32:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7d1f371740 1 -- 172.21.15.26:0/3131236820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c0fb60 msgr2=0x55daf3c18a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7d1f371740 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c0fb60 0x55daf3c18a20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c005e30 tx=0x7f7d0c004a80 comp rx=0 tx=0).stop 2024-04-21T10:32:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7d1f371740 1 -- 172.21.15.26:0/3131236820 shutdown_connections 2024-04-21T10:32:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7d1f371740 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55daf3c29090 0x55daf3c2b4e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7d1f371740 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7d1f371740 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c108d0 0x55daf3c15db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7d1f371740 1 --2- 172.21.15.26:0/3131236820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c0fb60 0x55daf3c18a20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7d1f371740 1 -- 172.21.15.26:0/3131236820 >> 172.21.15.26:0/3131236820 conn(0x55daf3bd6c90 msgr2=0x55daf3c0c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7cfdffb640 0 client.15780 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7d1f371740 1 -- 172.21.15.26:0/3131236820 shutdown_connections 2024-04-21T10:32:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.975+0000 7f7d1f371740 1 -- 172.21.15.26:0/3131236820 wait complete. 2024-04-21T10:32:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.976+0000 7f7d1f371740 20 client.15780 trim_cache size 0 max 16384 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.991+0000 7f7d1f371740 1 Processor -- start 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.991+0000 7f7d1f371740 1 -- start start 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.991+0000 7f7d1f371740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c0fb60 0x55daf3c24d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.991+0000 7f7d1f371740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c2b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.991+0000 7f7d1f371740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c190e0 0x55daf3c184e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.991+0000 7f7d1f371740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141b04a0 con 0x55daf3c108d0 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.991+0000 7f7d1f371740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141add10 con 0x55daf3c0fb60 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.991+0000 7f7d1f371740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141abbc0 con 0x55daf3c190e0 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.991+0000 7f7d1a441640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c2b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.991+0000 7f7d1a441640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c2b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39868/0 (socket says 172.21.15.26:39868) 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.991+0000 7f7d1a441640 1 -- 172.21.15.26:0/333646181 learned_addr learned my addr 172.21.15.26:0/333646181 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.991+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/333646181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c190e0 0x55daf3c184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.992+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/333646181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c0fb60 0x55daf3c24d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.992+0000 7f7d1a441640 1 -- 172.21.15.26:0/333646181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c190e0 msgr2=0x55daf3c184e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.992+0000 7f7d1a441640 1 --2- 172.21.15.26:0/333646181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c190e0 0x55daf3c184e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.992+0000 7f7d1a441640 1 -- 172.21.15.26:0/333646181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c0fb60 msgr2=0x55daf3c24d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.992+0000 7f7d1a441640 1 --2- 172.21.15.26:0/333646181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c0fb60 0x55daf3c24d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.992+0000 7f7d1a441640 1 -- 172.21.15.26:0/333646181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf3c14300 con 0x55daf3c108d0 2024-04-21T10:32:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.992+0000 7f7d1a441640 1 --2- 172.21.15.26:0/333646181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c2b0a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009c00 tx=0x7f7d10009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6faa50bdee8e4bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.992+0000 7f7cfffff640 1 -- 172.21.15.26:0/333646181 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1001e040 con 0x55daf3c108d0 2024-04-21T10:32:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.992+0000 7f7cfffff640 1 -- 172.21.15.26:0/333646181 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d1001f040 con 0x55daf3c108d0 2024-04-21T10:32:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.992+0000 7f7cfffff640 1 -- 172.21.15.26:0/333646181 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10019650 con 0x55daf3c108d0 2024-04-21T10:32:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.993+0000 7f7d1f371740 1 -- 172.21.15.26:0/333646181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 msgr2=0x55daf3c2b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.993+0000 7f7d1f371740 1 --2- 172.21.15.26:0/333646181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c2b0a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009c00 tx=0x7f7d10009b60 comp rx=0 tx=0).stop 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.993+0000 7f7d1f371740 1 -- 172.21.15.26:0/333646181 shutdown_connections 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.993+0000 7f7d1f371740 1 --2- 172.21.15.26:0/333646181 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c190e0 0x55daf3c184e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.993+0000 7f7d1f371740 1 --2- 172.21.15.26:0/333646181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c2b0a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.993+0000 7f7d1f371740 1 --2- 172.21.15.26:0/333646181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c0fb60 0x55daf3c24d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.993+0000 7f7d1f371740 1 -- 172.21.15.26:0/333646181 >> 172.21.15.26:0/333646181 conn(0x55daf3bd6c90 msgr2=0x55daf3c11060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.993+0000 7f7d1f371740 1 -- 172.21.15.26:0/333646181 shutdown_connections 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.993+0000 7f7d1f371740 1 -- 172.21.15.26:0/333646181 wait complete. 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.993+0000 7f7d1f371740 1 -- start start 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.993+0000 7f7d1f371740 1 Processor -- start 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1f371740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c0fb60 0x55daf3c194c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1f371740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c19a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1f371740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c19f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1f371740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141ac1d0 con 0x55daf3c108d0 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1f371740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141ad640 con 0x55daf3c190e0 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1f371740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141ac8a0 con 0x55daf3c0fb60 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1f371740 10 client.0 _ll_register_callbacks cb 0x7f7d08005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1a441640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c19a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1ac42640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c0fb60 0x55daf3c194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1a441640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c19a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39872/0 (socket says 172.21.15.26:39872) 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1a441640 1 -- 172.21.15.26:0/1727457580 learned_addr learned my addr 172.21.15.26:0/1727457580 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c19f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1a441640 1 -- 172.21.15.26:0/1727457580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c0fb60 msgr2=0x55daf3c194c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1a441640 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c0fb60 0x55daf3c194c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1a441640 1 -- 172.21.15.26:0/1727457580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 msgr2=0x55daf3c19f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1a441640 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c19f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1a441640 1 -- 172.21.15.26:0/1727457580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf3c204c0 con 0x55daf3c108d0 2024-04-21T10:32:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c19f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7d1a441640 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c19a00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009f10 tx=0x7f7d10009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c93fe5d5df7d5b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7cfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7cfdffb640 1 -- 172.21.15.26:0/1727457580 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10026440 con 0x55daf3c108d0 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.994+0000 7f7cfdffb640 1 -- 172.21.15.26:0/1727457580 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f7d100265e0 con 0x55daf3c108d0 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.995+0000 7f7d1f371740 1 -- 172.21.15.26:0/1727457580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d1001f040 con 0x55daf3c108d0 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.995+0000 7f7cfdffb640 1 -- 172.21.15.26:0/1727457580 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7d100267e0 con 0x55daf3c108d0 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.995+0000 7f7d1f371740 10 client.15786 fetch_fsmap learned FSMap version 24 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.995+0000 7f7d1f371740 1 -- 172.21.15.26:0/1727457580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55daf3c21790 con 0x55daf3c108d0 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.995+0000 7f7cfdffb640 1 -- 172.21.15.26:0/1727457580 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d1001f040 con 0x55daf3c108d0 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.996+0000 7f7cfdffb640 1 -- 172.21.15.26:0/1727457580 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f7d10024040 con 0x55daf3c108d0 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.996+0000 7f7d1f371740 10 client.15786 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.996+0000 7f7d1f371740 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.996+0000 7f7d1f371740 20 client.15786 populate_metadata read hostname 'smithi026' 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.996+0000 7f7d1f371740 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55daf3c8f660 0x55daf3c91a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.996+0000 7f7d1f371740 4 client.15786 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T10:32:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.996+0000 7f7d1f371740 1 -- 172.21.15.26:0/1727457580 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55daf3c2b870 con 0x55daf3c8f660 2024-04-21T10:32:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.997+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55daf3c8f660 0x55daf3c91a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.998+0000 7f7cfdffb640 10 client.15786 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:32.998+0000 7f7d1ac42640 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55daf3c8f660 0x55daf3c91a50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37992398ef7c518e server_cookie=f304adb5a441e260 in_seq=0 out_seq=0 2024-04-21T10:32:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7cfdffb640 1 -- 172.21.15.26:0/1727457580 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55daf3c2b870 con 0x55daf3c8f660 2024-04-21T10:32:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7cfdffb640 10 client.15786 handle_command_reply: tid=0 2024-04-21T10:32:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7d1f371740 1 client.15786 shutdown 2024-04-21T10:32:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7d1f371740 1 -- 172.21.15.26:0/1727457580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 msgr2=0x55daf3c19a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7d1f371740 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c19a00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009f10 tx=0x7f7d10009680 comp rx=0 tx=0).stop 2024-04-21T10:32:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7d1f371740 1 -- 172.21.15.26:0/1727457580 shutdown_connections 2024-04-21T10:32:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7d1f371740 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55daf3c8f660 0x55daf3c91a50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7d1f371740 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55daf3c190e0 0x55daf3c19f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7d1f371740 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55daf3c108d0 0x55daf3c19a00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7d1f371740 1 --2- 172.21.15.26:0/1727457580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55daf3c0fb60 0x55daf3c194c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7cfdffb640 0 client.15786 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7d1f371740 1 -- 172.21.15.26:0/1727457580 >> 172.21.15.26:0/1727457580 conn(0x55daf3bd6c90 msgr2=0x55daf3c145b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7d1f371740 1 -- 172.21.15.26:0/1727457580 shutdown_connections 2024-04-21T10:32:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.279+0000 7f7d1f371740 1 -- 172.21.15.26:0/1727457580 wait complete. 2024-04-21T10:32:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1f371740 20 client.15786 trim_cache size 0 max 16384 2024-04-21T10:32:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-04-21T10:32:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-04-21T10:32:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-04-21T10:32:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-04-21T10:32:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2715277745 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f7d000a3dc0 msgr2=0x7f7d000a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1c6a5640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f7d000a3dc0 0x7f7d000a6280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d080ce420 tx=0x7f7d080049b0 comp rx=0 tx=0).stop 2024-04-21T10:32:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2715277745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d14106190 msgr2=0x7f7d14198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1c6a5640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d14106190 0x7f7d14198bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800e9c0 tx=0x7f7d0800ee80 comp rx=0 tx=0).stop 2024-04-21T10:32:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2715277745 shutdown_connections 2024-04-21T10:32:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1c6a5640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f7d000a3dc0 0x7f7d000a6280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1c6a5640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d1419cf70 0x7f7d1419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1c6a5640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d14106190 0x7f7d14198bf0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1c6a5640 1 --2- 172.21.15.26:0/2715277745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d141010c0 0x7f7d141986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2715277745 >> 172.21.15.26:0/2715277745 conn(0x7f7d14077f60 msgr2=0x7f7d1406a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2715277745 shutdown_connections 2024-04-21T10:32:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:33.280+0000 7f7d1c6a5640 1 -- 172.21.15.26:0/2715277745 wait complete. 2024-04-21T10:32:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:33 smithi026 ceph-mon[27965]: pgmap v50: 97 pgs: 97 active+clean; 48 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 754 op/s 2024-04-21T10:32:33.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:33 smithi169 ceph-mon[31469]: pgmap v50: 97 pgs: 97 active+clean; 48 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 754 op/s 2024-04-21T10:32:33.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:33 smithi190 ceph-mon[31423]: pgmap v50: 97 pgs: 97 active+clean; 48 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 754 op/s 2024-04-21T10:32:33.818 INFO:tasks.workunit.client.0.smithi026.stdout:autoheader 2024-04-21T10:32:34.142 INFO:tasks.workunit.client.0.smithi026.stdout:autoconf 2024-04-21T10:32:34.341 INFO:tasks.workunit.client.0.smithi026.stdout:./configure \ 2024-04-21T10:32:34.342 INFO:tasks.workunit.client.0.smithi026.stdout: --libexecdir=/usr/lib \ 2024-04-21T10:32:34.342 INFO:tasks.workunit.client.0.smithi026.stdout: --exec_prefix=/var/lib 2024-04-21T10:32:34.490 INFO:tasks.workunit.client.0.smithi026.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-21T10:32:34.490 INFO:tasks.workunit.client.0.smithi026.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-21T10:32:34.491 INFO:tasks.workunit.client.0.smithi026.stdout:checking how to print strings... printf 2024-04-21T10:32:34.491 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gcc... gcc 2024-04-21T10:32:34.578 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether the C compiler works... yes 2024-04-21T10:32:34.578 INFO:tasks.workunit.client.0.smithi026.stdout:checking for C compiler default output file name... a.out 2024-04-21T10:32:34.617 INFO:tasks.workunit.client.0.smithi026.stdout:checking for suffix of executables... 2024-04-21T10:32:34.655 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether we are cross compiling... no 2024-04-21T10:32:34.694 INFO:tasks.workunit.client.0.smithi026.stdout:checking for suffix of object files... o 2024-04-21T10:32:34.718 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether we are using the GNU C compiler... yes 2024-04-21T10:32:34.744 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether gcc accepts -g... yes 2024-04-21T10:32:34.816 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-21T10:32:34.820 INFO:tasks.workunit.client.0.smithi026.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-21T10:32:34.825 INFO:tasks.workunit.client.0.smithi026.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-21T10:32:34.826 INFO:tasks.workunit.client.0.smithi026.stdout:checking for egrep... /usr/bin/grep -E 2024-04-21T10:32:34.827 INFO:tasks.workunit.client.0.smithi026.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-21T10:32:34.830 INFO:tasks.workunit.client.0.smithi026.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-21T10:32:34.832 INFO:tasks.workunit.client.0.smithi026.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-21T10:32:34.843 INFO:tasks.workunit.client.0.smithi026.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-21T10:32:35.646 INFO:tasks.workunit.client.0.smithi026.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-21T10:32:35.647 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether ln -s works... yes 2024-04-21T10:32:35.656 INFO:tasks.workunit.client.0.smithi026.stdout:checking the maximum length of command line arguments... 1572864 2024-04-21T10:32:35.660 INFO:tasks.workunit.client.0.smithi026.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-21T10:32:35.660 INFO:tasks.workunit.client.0.smithi026.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-21T10:32:35.660 INFO:tasks.workunit.client.0.smithi026.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-21T10:32:35.660 INFO:tasks.workunit.client.0.smithi026.stdout:checking for objdump... objdump 2024-04-21T10:32:35.660 INFO:tasks.workunit.client.0.smithi026.stdout:checking how to recognize dependent libraries... pass_all 2024-04-21T10:32:35.660 INFO:tasks.workunit.client.0.smithi026.stdout:checking for dlltool... no 2024-04-21T10:32:35.660 INFO:tasks.workunit.client.0.smithi026.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-21T10:32:35.660 INFO:tasks.workunit.client.0.smithi026.stdout:checking for ar... ar 2024-04-21T10:32:35.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:35 smithi026 ceph-mon[27965]: pgmap v51: 97 pgs: 97 active+clean; 48 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 754 op/s 2024-04-21T10:32:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:35 smithi169 ceph-mon[31469]: pgmap v51: 97 pgs: 97 active+clean; 48 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 754 op/s 2024-04-21T10:32:35.747 INFO:tasks.workunit.client.0.smithi026.stdout:checking for archiver @FILE support... @ 2024-04-21T10:32:35.747 INFO:tasks.workunit.client.0.smithi026.stdout:checking for strip... strip 2024-04-21T10:32:35.747 INFO:tasks.workunit.client.0.smithi026.stdout:checking for ranlib... ranlib 2024-04-21T10:32:35.747 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gawk... gawk 2024-04-21T10:32:35.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:35 smithi190 ceph-mon[31423]: pgmap v51: 97 pgs: 97 active+clean; 48 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 754 op/s 2024-04-21T10:32:35.854 INFO:tasks.workunit.client.0.smithi026.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-21T10:32:35.855 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sysroot... no 2024-04-21T10:32:35.867 INFO:tasks.workunit.client.0.smithi026.stdout:checking for a working dd... /usr/bin/dd 2024-04-21T10:32:35.877 INFO:tasks.workunit.client.0.smithi026.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-21T10:32:35.911 INFO:tasks.workunit.client.0.smithi026.stdout:checking for mt... no 2024-04-21T10:32:35.924 INFO:tasks.workunit.client.0.smithi026.stdout:checking if : is a manifest tool... no 2024-04-21T10:32:36.019 INFO:tasks.workunit.client.0.smithi026.stdout:checking how to run the C preprocessor... gcc -E 2024-04-21T10:32:36.217 INFO:tasks.workunit.client.0.smithi026.stdout:checking for ANSI C header files... yes 2024-04-21T10:32:36.254 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/types.h... yes 2024-04-21T10:32:36.294 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/stat.h... yes 2024-04-21T10:32:36.333 INFO:tasks.workunit.client.0.smithi026.stdout:checking for stdlib.h... yes 2024-04-21T10:32:36.374 INFO:tasks.workunit.client.0.smithi026.stdout:checking for string.h... yes 2024-04-21T10:32:36.414 INFO:tasks.workunit.client.0.smithi026.stdout:checking for memory.h... yes 2024-04-21T10:32:36.454 INFO:tasks.workunit.client.0.smithi026.stdout:checking for strings.h... yes 2024-04-21T10:32:36.494 INFO:tasks.workunit.client.0.smithi026.stdout:checking for inttypes.h... yes 2024-04-21T10:32:36.534 INFO:tasks.workunit.client.0.smithi026.stdout:checking for stdint.h... yes 2024-04-21T10:32:36.576 INFO:tasks.workunit.client.0.smithi026.stdout:checking for unistd.h... yes 2024-04-21T10:32:36.619 INFO:tasks.workunit.client.0.smithi026.stdout:checking for dlfcn.h... yes 2024-04-21T10:32:36.625 INFO:tasks.workunit.client.0.smithi026.stdout:checking for objdir... .libs 2024-04-21T10:32:36.752 INFO:tasks.workunit.client.0.smithi026.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-21T10:32:36.752 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-21T10:32:36.794 INFO:tasks.workunit.client.0.smithi026.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-21T10:32:36.841 INFO:tasks.workunit.client.0.smithi026.stdout:checking if gcc static flag -static works... no 2024-04-21T10:32:36.910 INFO:tasks.workunit.client.0.smithi026.stdout:checking if gcc supports -c -o file.o... yes 2024-04-21T10:32:36.910 INFO:tasks.workunit.client.0.smithi026.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-21T10:32:36.916 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-21T10:32:36.965 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether -lc should be explicitly linked in... no 2024-04-21T10:32:37.026 INFO:tasks.workunit.client.0.smithi026.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-21T10:32:37.026 INFO:tasks.workunit.client.0.smithi026.stdout:checking how to hardcode library paths into programs... immediate 2024-04-21T10:32:37.027 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether stripping libraries is possible... yes 2024-04-21T10:32:37.027 INFO:tasks.workunit.client.0.smithi026.stdout:checking if libtool supports shared libraries... yes 2024-04-21T10:32:37.027 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether to build shared libraries... yes 2024-04-21T10:32:37.027 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether to build static libraries... yes 2024-04-21T10:32:37.033 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gcc... (cached) gcc 2024-04-21T10:32:37.070 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-21T10:32:37.070 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-21T10:32:37.070 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-21T10:32:37.070 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gmake... /usr/bin/gmake 2024-04-21T10:32:37.071 INFO:tasks.workunit.client.0.smithi026.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-04-21T10:32:37.071 INFO:tasks.workunit.client.0.smithi026.stdout:checking for tar... /bin/tar 2024-04-21T10:32:37.071 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gzip... /bin/gzip 2024-04-21T10:32:37.099 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether gcc -MM is supported... yes 2024-04-21T10:32:37.099 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sort... /bin/sort 2024-04-21T10:32:37.099 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether ln -s works... yes 2024-04-21T10:32:37.099 INFO:tasks.workunit.client.0.smithi026.stdout:checking for rpm... /bin/rpm 2024-04-21T10:32:37.102 INFO:tasks.workunit.client.0.smithi026.stdout:checking for rpmbuild... no 2024-04-21T10:32:37.103 INFO:tasks.workunit.client.0.smithi026.stdout:checking for ANSI C header files... (cached) yes 2024-04-21T10:32:37.174 INFO:tasks.workunit.client.0.smithi026.stdout:checking assert.h usability... yes 2024-04-21T10:32:37.192 INFO:tasks.workunit.client.0.smithi026.stdout:checking assert.h presence... yes 2024-04-21T10:32:37.192 INFO:tasks.workunit.client.0.smithi026.stdout:checking for assert.h... yes 2024-04-21T10:32:37.249 INFO:tasks.workunit.client.0.smithi026.stdout:checking bstring.h usability... no 2024-04-21T10:32:37.279 INFO:tasks.workunit.client.0.smithi026.stdout:checking bstring.h presence... no 2024-04-21T10:32:37.279 INFO:tasks.workunit.client.0.smithi026.stdout:checking for bstring.h... no 2024-04-21T10:32:37.326 INFO:tasks.workunit.client.0.smithi026.stdout:checking libgen.h usability... yes 2024-04-21T10:32:37.343 INFO:tasks.workunit.client.0.smithi026.stdout:checking libgen.h presence... yes 2024-04-21T10:32:37.343 INFO:tasks.workunit.client.0.smithi026.stdout:checking for libgen.h... yes 2024-04-21T10:32:37.398 INFO:tasks.workunit.client.0.smithi026.stdout:checking dirent.h usability... yes 2024-04-21T10:32:37.416 INFO:tasks.workunit.client.0.smithi026.stdout:checking dirent.h presence... yes 2024-04-21T10:32:37.416 INFO:tasks.workunit.client.0.smithi026.stdout:checking for dirent.h... yes 2024-04-21T10:32:37.466 INFO:tasks.workunit.client.0.smithi026.stdout:checking errno.h usability... yes 2024-04-21T10:32:37.483 INFO:tasks.workunit.client.0.smithi026.stdout:checking errno.h presence... yes 2024-04-21T10:32:37.483 INFO:tasks.workunit.client.0.smithi026.stdout:checking for errno.h... yes 2024-04-21T10:32:37.542 INFO:tasks.workunit.client.0.smithi026.stdout:checking malloc.h usability... yes 2024-04-21T10:32:37.565 INFO:tasks.workunit.client.0.smithi026.stdout:checking malloc.h presence... yes 2024-04-21T10:32:37.565 INFO:tasks.workunit.client.0.smithi026.stdout:checking for malloc.h... yes 2024-04-21T10:32:37.622 INFO:tasks.workunit.client.0.smithi026.stdout:checking uuid.h usability... no 2024-04-21T10:32:37.655 INFO:tasks.workunit.client.0.smithi026.stdout:checking uuid.h presence... no 2024-04-21T10:32:37.655 INFO:tasks.workunit.client.0.smithi026.stdout:checking for uuid.h... no 2024-04-21T10:32:37.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:37 smithi169 ceph-mon[31469]: pgmap v52: 97 pgs: 97 active+clean; 47 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 1.05k op/s 2024-04-21T10:32:37.698 INFO:tasks.workunit.client.0.smithi026.stdout:checking uuid/uuid.h usability... yes 2024-04-21T10:32:37.717 INFO:tasks.workunit.client.0.smithi026.stdout:checking uuid/uuid.h presence... yes 2024-04-21T10:32:37.717 INFO:tasks.workunit.client.0.smithi026.stdout:checking for uuid/uuid.h... yes 2024-04-21T10:32:37.769 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/uuid.h usability... no 2024-04-21T10:32:37.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:37 smithi190 ceph-mon[31423]: pgmap v52: 97 pgs: 97 active+clean; 47 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 1.05k op/s 2024-04-21T10:32:37.799 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/uuid.h presence... no 2024-04-21T10:32:37.799 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/uuid.h... no 2024-04-21T10:32:37.858 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/file.h usability... yes 2024-04-21T10:32:37.878 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/file.h presence... yes 2024-04-21T10:32:37.878 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/file.h... yes 2024-04-21T10:32:37.930 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/fcntl.h usability... yes 2024-04-21T10:32:37.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:37 smithi026 ceph-mon[27965]: pgmap v52: 97 pgs: 97 active+clean; 47 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 1.05k op/s 2024-04-21T10:32:37.948 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/fcntl.h presence... yes 2024-04-21T10:32:37.949 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/fcntl.h... yes 2024-04-21T10:32:38.005 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/syssgi.h usability... no 2024-04-21T10:32:38.035 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/syssgi.h presence... no 2024-04-21T10:32:38.035 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/syssgi.h... no 2024-04-21T10:32:38.201 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/param.h usability... yes 2024-04-21T10:32:38.222 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/param.h presence... yes 2024-04-21T10:32:38.222 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/param.h... yes 2024-04-21T10:32:38.238 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/stat.h... (cached) yes 2024-04-21T10:32:38.281 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/statvfs.h usability... yes 2024-04-21T10:32:38.303 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/statvfs.h presence... yes 2024-04-21T10:32:38.303 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/statvfs.h... yes 2024-04-21T10:32:38.352 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/time.h usability... yes 2024-04-21T10:32:38.370 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/time.h presence... yes 2024-04-21T10:32:38.370 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/time.h... yes 2024-04-21T10:32:38.417 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/ioctl.h usability... yes 2024-04-21T10:32:38.435 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/ioctl.h presence... yes 2024-04-21T10:32:38.435 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/ioctl.h... yes 2024-04-21T10:32:38.492 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/wait.h usability... yes 2024-04-21T10:32:38.515 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/wait.h presence... yes 2024-04-21T10:32:38.515 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/wait.h... yes 2024-04-21T10:32:38.530 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/types.h... (cached) yes 2024-04-21T10:32:38.535 INFO:tasks.workunit.client.0.smithi026.stdout:checking for strings.h... (cached) yes 2024-04-21T10:32:38.572 INFO:tasks.workunit.client.0.smithi026.stdout:checking err.h usability... yes 2024-04-21T10:32:38.589 INFO:tasks.workunit.client.0.smithi026.stdout:checking err.h presence... yes 2024-04-21T10:32:38.589 INFO:tasks.workunit.client.0.smithi026.stdout:checking for err.h... yes 2024-04-21T10:32:38.636 INFO:tasks.workunit.client.0.smithi026.stdout:checking linux/falloc.h usability... yes 2024-04-21T10:32:38.652 INFO:tasks.workunit.client.0.smithi026.stdout:checking linux/falloc.h presence... yes 2024-04-21T10:32:38.652 INFO:tasks.workunit.client.0.smithi026.stdout:checking for linux/falloc.h... yes 2024-04-21T10:32:38.705 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-21T10:32:38.735 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-21T10:32:38.735 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-21T10:32:38.779 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-21T10:32:38.808 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-21T10:32:38.808 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-21T10:32:38.854 INFO:tasks.workunit.client.0.smithi026.stdout:checking xfs/platform_defs.h usability... no 2024-04-21T10:32:38.883 INFO:tasks.workunit.client.0.smithi026.stdout:checking xfs/platform_defs.h presence... no 2024-04-21T10:32:38.883 INFO:tasks.workunit.client.0.smithi026.stdout:checking for xfs/platform_defs.h... no 2024-04-21T10:32:38.930 INFO:tasks.workunit.client.0.smithi026.stdout:checking btrfs/ioctl.h usability... no 2024-04-21T10:32:38.961 INFO:tasks.workunit.client.0.smithi026.stdout:checking btrfs/ioctl.h presence... no 2024-04-21T10:32:38.961 INFO:tasks.workunit.client.0.smithi026.stdout:checking for btrfs/ioctl.h... no 2024-04-21T10:32:39.006 INFO:tasks.workunit.client.0.smithi026.stdout:checking cifs/ioctl.h usability... no 2024-04-21T10:32:39.036 INFO:tasks.workunit.client.0.smithi026.stdout:checking cifs/ioctl.h presence... no 2024-04-21T10:32:39.036 INFO:tasks.workunit.client.0.smithi026.stdout:checking for cifs/ioctl.h... no 2024-04-21T10:32:39.086 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/mman.h usability... yes 2024-04-21T10:32:39.104 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/mman.h presence... yes 2024-04-21T10:32:39.104 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/mman.h... yes 2024-04-21T10:32:39.143 INFO:tasks.workunit.client.0.smithi026.stdout:checking for xfs/xfs_log_format.h... no 2024-04-21T10:32:39.199 INFO:tasks.workunit.client.0.smithi026.stdout:checking for uuid_compare... no 2024-04-21T10:32:39.240 INFO:tasks.workunit.client.0.smithi026.stdout:checking for uuid_compare in -luuid... yes 2024-04-21T10:32:39.295 INFO:tasks.workunit.client.0.smithi026.stdout:checking for xfs/xfs.h... yes 2024-04-21T10:32:39.330 INFO:tasks.workunit.client.0.smithi026.stdout:checking for xfs/libxfs.h... no 2024-04-21T10:32:39.359 INFO:tasks.workunit.client.0.smithi026.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-21T10:32:39.398 INFO:tasks.workunit.client.0.smithi026.stdout:checking for xfs/xqm.h... yes 2024-04-21T10:32:39.452 INFO:tasks.workunit.client.0.smithi026.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-21T10:32:39.488 INFO:tasks.workunit.client.0.smithi026.stdout:checking xfs/handle.h usability... yes 2024-04-21T10:32:39.504 INFO:tasks.workunit.client.0.smithi026.stdout:checking xfs/handle.h presence... yes 2024-04-21T10:32:39.504 INFO:tasks.workunit.client.0.smithi026.stdout:checking for xfs/handle.h... yes 2024-04-21T10:32:39.545 INFO:tasks.workunit.client.0.smithi026.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-21T10:32:39.605 INFO:tasks.workunit.client.0.smithi026.stdout:checking attr/attributes.h usability... yes 2024-04-21T10:32:39.622 INFO:tasks.workunit.client.0.smithi026.stdout:checking attr/attributes.h presence... yes 2024-04-21T10:32:39.622 INFO:tasks.workunit.client.0.smithi026.stdout:checking for attr/attributes.h... yes 2024-04-21T10:32:39.673 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/attributes.h usability... no 2024-04-21T10:32:39.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:39 smithi169 ceph-mon[31469]: pgmap v53: 97 pgs: 97 active+clean; 47 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 592 op/s 2024-04-21T10:32:39.702 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/attributes.h presence... no 2024-04-21T10:32:39.702 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/attributes.h... no 2024-04-21T10:32:39.742 INFO:tasks.workunit.client.0.smithi026.stdout:checking for attr_list in -lattr... yes 2024-04-21T10:32:39.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:39 smithi190 ceph-mon[31423]: pgmap v53: 97 pgs: 97 active+clean; 47 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 592 op/s 2024-04-21T10:32:39.783 INFO:tasks.workunit.client.0.smithi026.stdout:checking for attr_set in -lattr... yes 2024-04-21T10:32:39.825 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/acl.h usability... yes 2024-04-21T10:32:39.842 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/acl.h presence... yes 2024-04-21T10:32:39.843 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/acl.h... yes 2024-04-21T10:32:39.876 INFO:tasks.workunit.client.0.smithi026.stdout:checking acl/libacl.h usability... yes 2024-04-21T10:32:39.894 INFO:tasks.workunit.client.0.smithi026.stdout:checking acl/libacl.h presence... yes 2024-04-21T10:32:39.895 INFO:tasks.workunit.client.0.smithi026.stdout:checking for acl/libacl.h... yes 2024-04-21T10:32:39.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:39 smithi026 ceph-mon[27965]: pgmap v53: 97 pgs: 97 active+clean; 47 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 592 op/s 2024-04-21T10:32:39.936 INFO:tasks.workunit.client.0.smithi026.stdout:checking for acl_init in -lacl... yes 2024-04-21T10:32:40.000 INFO:tasks.workunit.client.0.smithi026.stdout:checking gdbm-ndbm.h usability... no 2024-04-21T10:32:40.030 INFO:tasks.workunit.client.0.smithi026.stdout:checking gdbm-ndbm.h presence... no 2024-04-21T10:32:40.030 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gdbm-ndbm.h... no 2024-04-21T10:32:40.070 INFO:tasks.workunit.client.0.smithi026.stdout:checking gdbm/ndbm.h usability... no 2024-04-21T10:32:40.097 INFO:tasks.workunit.client.0.smithi026.stdout:checking gdbm/ndbm.h presence... no 2024-04-21T10:32:40.098 INFO:tasks.workunit.client.0.smithi026.stdout:checking for gdbm/ndbm.h... no 2024-04-21T10:32:40.136 INFO:tasks.workunit.client.0.smithi026.stdout:checking ndbm.h usability... no 2024-04-21T10:32:40.166 INFO:tasks.workunit.client.0.smithi026.stdout:checking ndbm.h presence... no 2024-04-21T10:32:40.166 INFO:tasks.workunit.client.0.smithi026.stdout:checking for ndbm.h... no 2024-04-21T10:32:40.199 INFO:tasks.workunit.client.0.smithi026.stdout:checking libaio.h usability... yes 2024-04-21T10:32:40.218 INFO:tasks.workunit.client.0.smithi026.stdout:checking libaio.h presence... yes 2024-04-21T10:32:40.219 INFO:tasks.workunit.client.0.smithi026.stdout:checking for libaio.h... yes 2024-04-21T10:32:40.271 INFO:tasks.workunit.client.0.smithi026.stdout:checking for dm_make_handle in -ldm... no 2024-04-21T10:32:40.271 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-04-21T10:32:40.271 INFO:tasks.workunit.client.0.smithi026.stdout:WARNING: could not find a valid DMAPI base library. 2024-04-21T10:32:40.271 INFO:tasks.workunit.client.0.smithi026.stdout:If you want DMAPI support please install the data migration 2024-04-21T10:32:40.272 INFO:tasks.workunit.client.0.smithi026.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-04-21T10:32:40.272 INFO:tasks.workunit.client.0.smithi026.stdout:from the dmapi source. 2024-04-21T10:32:40.272 INFO:tasks.workunit.client.0.smithi026.stdout: 2024-04-21T10:32:40.305 INFO:tasks.workunit.client.0.smithi026.stdout:checking linux/fiemap.h usability... yes 2024-04-21T10:32:40.321 INFO:tasks.workunit.client.0.smithi026.stdout:checking linux/fiemap.h presence... yes 2024-04-21T10:32:40.321 INFO:tasks.workunit.client.0.smithi026.stdout:checking for linux/fiemap.h... yes 2024-04-21T10:32:40.359 INFO:tasks.workunit.client.0.smithi026.stdout:checking for fallocate... yes 2024-04-21T10:32:40.406 INFO:tasks.workunit.client.0.smithi026.stdout:checking for open_by_handle_at... yes 2024-04-21T10:32:40.446 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/prctl.h usability... yes 2024-04-21T10:32:40.464 INFO:tasks.workunit.client.0.smithi026.stdout:checking sys/prctl.h presence... yes 2024-04-21T10:32:40.464 INFO:tasks.workunit.client.0.smithi026.stdout:checking for sys/prctl.h... yes 2024-04-21T10:32:40.498 INFO:tasks.workunit.client.0.smithi026.stdout:checking linux/fs.h usability... yes 2024-04-21T10:32:40.515 INFO:tasks.workunit.client.0.smithi026.stdout:checking linux/fs.h presence... yes 2024-04-21T10:32:40.515 INFO:tasks.workunit.client.0.smithi026.stdout:checking for linux/fs.h... yes 2024-04-21T10:32:40.577 INFO:tasks.workunit.client.0.smithi026.stdout:checking btrfsutil.h usability... no 2024-04-21T10:32:40.608 INFO:tasks.workunit.client.0.smithi026.stdout:checking btrfsutil.h presence... no 2024-04-21T10:32:40.608 INFO:tasks.workunit.client.0.smithi026.stdout:checking for btrfsutil.h... no 2024-04-21T10:32:40.659 INFO:tasks.workunit.client.0.smithi026.stdout:checking for copy_file_range... yes 2024-04-21T10:32:40.707 INFO:tasks.workunit.client.0.smithi026.stdout:checking for renameat2... yes 2024-04-21T10:32:40.727 INFO:tasks.workunit.client.0.smithi026.stdout:configure: creating ./config.status 2024-04-21T10:32:41.060 INFO:tasks.workunit.client.0.smithi026.stdout:config.status: creating include/builddefs 2024-04-21T10:32:41.074 INFO:tasks.workunit.client.0.smithi026.stdout:config.status: creating include/config.h 2024-04-21T10:32:41.086 INFO:tasks.workunit.client.0.smithi026.stdout:config.status: executing libtool commands 2024-04-21T10:32:41.176 INFO:tasks.workunit.client.0.smithi026.stdout:Building include 2024-04-21T10:32:41.176 INFO:tasks.workunit.client.0.smithi026.stdout:Building lib 2024-04-21T10:32:41.176 INFO:tasks.workunit.client.0.smithi026.stdout:Building tests 2024-04-21T10:32:41.181 INFO:tasks.workunit.client.0.smithi026.stdout: [LTDEP] 2024-04-21T10:32:41.349 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] dataascii.lo 2024-04-21T10:32:41.349 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] databin.lo 2024-04-21T10:32:41.349 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] datapid.lo 2024-04-21T10:32:41.349 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] file_lock.lo 2024-04-21T10:32:41.464 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] forker.lo 2024-04-21T10:32:41.476 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] pattern.lo 2024-04-21T10:32:41.509 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] open_flags.lo 2024-04-21T10:32:41.545 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] random_range.lo 2024-04-21T10:32:41.618 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] string_to_tokens.lo 2024-04-21T10:32:41.619 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] str_to_bytes.lo 2024-04-21T10:32:41.671 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] tlibio.lo 2024-04-21T10:32:41.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:41 smithi169 ceph-mon[31469]: pgmap v54: 97 pgs: 97 active+clean; 49 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 638 op/s 2024-04-21T10:32:41.729 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] write_log.lo 2024-04-21T10:32:41.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:41 smithi190 ceph-mon[31423]: pgmap v54: 97 pgs: 97 active+clean; 49 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 638 op/s 2024-04-21T10:32:41.790 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] random.lo 2024-04-21T10:32:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:41 smithi026 ceph-mon[27965]: pgmap v54: 97 pgs: 97 active+clean; 49 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 638 op/s 2024-04-21T10:32:41.972 INFO:tasks.workunit.client.0.smithi026.stdout: [LD] 2024-04-21T10:32:42.107 INFO:tasks.workunit.client.0.smithi026.stdout:Building ltp 2024-04-21T10:32:42.107 INFO:tasks.workunit.client.0.smithi026.stdout:Building src 2024-04-21T10:32:42.107 INFO:tasks.workunit.client.0.smithi026.stdout:Building m4 2024-04-21T10:32:42.107 INFO:tasks.workunit.client.0.smithi026.stdout:Building common 2024-04-21T10:32:42.115 INFO:tasks.workunit.client.0.smithi026.stdout: [DEP] 2024-04-21T10:32:42.133 INFO:tasks.workunit.client.0.smithi026.stdout: [DEP] 2024-04-21T10:32:42.333 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] doio 2024-04-21T10:32:42.334 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] fsstress 2024-04-21T10:32:42.334 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] fsx 2024-04-21T10:32:42.477 INFO:tasks.workunit.client.0.smithi026.stderr:fsstress.c: In function 'attr_list_path': 2024-04-21T10:32:42.477 INFO:tasks.workunit.client.0.smithi026.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-04-21T10:32:42.477 INFO:tasks.workunit.client.0.smithi026.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-04-21T10:32:42.477 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~ 2024-04-21T10:32:42.477 INFO:tasks.workunit.client.0.smithi026.stderr:In file included from ../src/global.h:25, 2024-04-21T10:32:42.477 INFO:tasks.workunit.client.0.smithi026.stderr: from fsstress.c:12: 2024-04-21T10:32:42.477 INFO:tasks.workunit.client.0.smithi026.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~~~~~ 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr:fsstress.c: In function 'attr_remove_path': 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~ 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr:In file included from ../src/global.h:25, 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr: from fsstress.c:12: 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~~~~~~~ 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr:fsstress.c: In function 'attr_set_path': 2024-04-21T10:32:42.478 INFO:tasks.workunit.client.0.smithi026.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-04-21T10:32:42.479 INFO:tasks.workunit.client.0.smithi026.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-04-21T10:32:42.479 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~ 2024-04-21T10:32:42.479 INFO:tasks.workunit.client.0.smithi026.stderr:In file included from ../src/global.h:25, 2024-04-21T10:32:42.479 INFO:tasks.workunit.client.0.smithi026.stderr: from fsstress.c:12: 2024-04-21T10:32:42.479 INFO:tasks.workunit.client.0.smithi026.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-04-21T10:32:42.479 INFO:tasks.workunit.client.0.smithi026.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-04-21T10:32:42.479 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~~~~ 2024-04-21T10:32:42.488 INFO:tasks.workunit.client.0.smithi026.stderr:fsstress.c: In function 'allocsp_f': 2024-04-21T10:32:42.488 INFO:tasks.workunit.client.0.smithi026.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-21T10:32:42.488 INFO:tasks.workunit.client.0.smithi026.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-04-21T10:32:42.488 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-21T10:32:42.488 INFO:tasks.workunit.client.0.smithi026.stderr: | XFS_IOC_RESVSP64 2024-04-21T10:32:42.488 INFO:tasks.workunit.client.0.smithi026.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-04-21T10:32:42.505 INFO:tasks.workunit.client.0.smithi026.stderr:fsstress.c: In function 'freesp_f': 2024-04-21T10:32:42.506 INFO:tasks.workunit.client.0.smithi026.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-21T10:32:42.506 INFO:tasks.workunit.client.0.smithi026.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-04-21T10:32:42.506 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-21T10:32:42.506 INFO:tasks.workunit.client.0.smithi026.stderr: | XFS_IOC_RESVSP64 2024-04-21T10:32:42.538 INFO:tasks.workunit.client.0.smithi026.stderr:gmake[3]: *** [Makefile:47: fsstress] Error 1 2024-04-21T10:32:42.538 INFO:tasks.workunit.client.0.smithi026.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-21T10:32:42.891 INFO:tasks.workunit.client.0.smithi026.stderr:doio.c: In function 'parse_cmdline': 2024-04-21T10:32:42.891 INFO:tasks.workunit.client.0.smithi026.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-04-21T10:32:42.891 INFO:tasks.workunit.client.0.smithi026.stderr: 3113 | fprintf(stderr, 2024-04-21T10:32:42.891 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~~~~~~~~~~~ 2024-04-21T10:32:42.891 INFO:tasks.workunit.client.0.smithi026.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-04-21T10:32:42.891 INFO:tasks.workunit.client.0.smithi026.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-21T10:32:42.891 INFO:tasks.workunit.client.0.smithi026.stderr: 3115 | Prog, TagName, tok); 2024-04-21T10:32:42.891 INFO:tasks.workunit.client.0.smithi026.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-04-21T10:32:42.947 INFO:tasks.workunit.client.0.smithi026.stderr:fsx.c: In function 'main': 2024-04-21T10:32:42.948 INFO:tasks.workunit.client.0.smithi026.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-04-21T10:32:42.948 INFO:tasks.workunit.client.0.smithi026.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-04-21T10:32:42.948 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-21T10:32:42.948 INFO:tasks.workunit.client.0.smithi026.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-04-21T10:32:42.948 INFO:tasks.workunit.client.0.smithi026.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-04-21T10:32:42.948 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-21T10:32:43.048 INFO:tasks.workunit.client.0.smithi026.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-04-21T10:32:43.048 INFO:tasks.workunit.client.0.smithi026.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-04-21T10:32:43.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:43 smithi169 ceph-mon[31469]: pgmap v55: 97 pgs: 97 active+clean; 49 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 342 op/s 2024-04-21T10:32:43.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:43 smithi190 ceph-mon[31423]: pgmap v55: 97 pgs: 97 active+clean; 49 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 342 op/s 2024-04-21T10:32:43.838 INFO:tasks.workunit.client.0.smithi026.stdout: [DEP] 2024-04-21T10:32:43.838 INFO:tasks.workunit.client.0.smithi026.stdout: [DEP] 2024-04-21T10:32:43.839 INFO:tasks.workunit.client.0.smithi026.stdout: [DEP] 2024-04-21T10:32:43.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:43 smithi026 ceph-mon[27965]: pgmap v55: 97 pgs: 97 active+clean; 49 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 342 op/s 2024-04-21T10:32:44.068 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] dirstress 2024-04-21T10:32:44.070 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] fill 2024-04-21T10:32:44.070 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] fill2 2024-04-21T10:32:44.071 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] getpagesize 2024-04-21T10:32:44.200 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] holes 2024-04-21T10:32:44.219 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] lstat64 2024-04-21T10:32:44.303 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] nametest 2024-04-21T10:32:44.321 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] permname 2024-04-21T10:32:44.396 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] randholes 2024-04-21T10:32:44.402 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] runas 2024-04-21T10:32:44.522 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] truncfile 2024-04-21T10:32:44.575 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] usemem 2024-04-21T10:32:44.595 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] mmapcat 2024-04-21T10:32:44.667 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] append_reader 2024-04-21T10:32:44.813 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] append_writer 2024-04-21T10:32:44.831 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] dirperf 2024-04-21T10:32:44.835 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] metaperf 2024-04-21T10:32:44.836 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] devzero 2024-04-21T10:32:44.968 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] feature 2024-04-21T10:32:45.044 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] alloc 2024-04-21T10:32:45.105 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] fault 2024-04-21T10:32:45.116 INFO:tasks.workunit.client.0.smithi026.stdout: [CC] fstest 2024-04-21T10:32:45.183 INFO:tasks.workunit.client.0.smithi026.stderr:alloc.c: In function 'main': 2024-04-21T10:32:45.184 INFO:tasks.workunit.client.0.smithi026.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-04-21T10:32:45.184 INFO:tasks.workunit.client.0.smithi026.stderr: 154 | bozo! 2024-04-21T10:32:45.184 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~ 2024-04-21T10:32:45.184 INFO:tasks.workunit.client.0.smithi026.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-04-21T10:32:45.184 INFO:tasks.workunit.client.0.smithi026.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-04-21T10:32:45.184 INFO:tasks.workunit.client.0.smithi026.stderr: 154 | bozo! 2024-04-21T10:32:45.184 INFO:tasks.workunit.client.0.smithi026.stderr: | ^ 2024-04-21T10:32:45.184 INFO:tasks.workunit.client.0.smithi026.stderr: | ; 2024-04-21T10:32:45.185 INFO:tasks.workunit.client.0.smithi026.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-04-21T10:32:45.185 INFO:tasks.workunit.client.0.smithi026.stderr: 184 | rflag = 1; 2024-04-21T10:32:45.185 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~ 2024-04-21T10:32:45.185 INFO:tasks.workunit.client.0.smithi026.stderr: | nflag 2024-04-21T10:32:45.185 INFO:tasks.workunit.client.0.smithi026.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-04-21T10:32:45.185 INFO:tasks.workunit.client.0.smithi026.stderr: 320 | bozo! 2024-04-21T10:32:45.185 INFO:tasks.workunit.client.0.smithi026.stderr: | ^ 2024-04-21T10:32:45.185 INFO:tasks.workunit.client.0.smithi026.stderr: | ; 2024-04-21T10:32:45.185 INFO:tasks.workunit.client.0.smithi026.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-04-21T10:32:45.185 INFO:tasks.workunit.client.0.smithi026.stderr: 128 | struct flock64 f; 2024-04-21T10:32:45.185 INFO:tasks.workunit.client.0.smithi026.stderr: | ^ 2024-04-21T10:32:45.190 INFO:tasks.workunit.client.0.smithi026.stderr:gmake[3]: *** [Makefile:83: alloc] Error 1 2024-04-21T10:32:45.190 INFO:tasks.workunit.client.0.smithi026.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-21T10:32:45.273 INFO:tasks.workunit.client.0.smithi026.stderr:fstest.c: In function 'run_child': 2024-04-21T10:32:45.273 INFO:tasks.workunit.client.0.smithi026.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-04-21T10:32:45.273 INFO:tasks.workunit.client.0.smithi026.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-21T10:32:45.273 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~ 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~~~~~~~ 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~ 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~~~~~~~ 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-21T10:32:45.274 INFO:tasks.workunit.client.0.smithi026.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-21T10:32:45.345 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-21T10:32:45.346 INFO:tasks.workunit.client.0.smithi026.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-04-21T10:32:45.346 INFO:tasks.workunit.client.0.smithi026.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-04-21T10:32:45.346 INFO:tasks.workunit.client.0.smithi026.stderr:make: *** [Makefile:51: default] Error 2 2024-04-21T10:32:45.346 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-21T10:32:45.346 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-21T10:32:45.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:45 smithi026 ceph-mon[27965]: pgmap v56: 97 pgs: 97 active+clean; 49 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 342 op/s 2024-04-21T10:32:45.697 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_b8ddeda347de6c45510a6df67440a1d298c37030/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph_b8ddeda347de6c45510a6df67440a1d298c37030/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_b8ddeda347de6c45510a6df67440a1d298c37030/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi026 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b8ddeda347de6c45510a6df67440a1d298c37030 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-21T10:32:45.928 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=127650b1201b47b19c9fd22f7fd5526f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_b8ddeda347de6c45510a6df67440a1d298c37030/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph_b8ddeda347de6c45510a6df67440a1d298c37030/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph_b8ddeda347de6c45510a6df67440a1d298c37030/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi026 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b8ddeda347de6c45510a6df67440a1d298c37030 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-21T10:32:45.931 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-21T10:32:45.941 INFO:tasks.quiescer:joining Quiescers 2024-04-21T10:32:45.941 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:32:45.946 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:45 smithi190 ceph-mon[31423]: pgmap v56: 97 pgs: 97 active+clean; 49 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 342 op/s 2024-04-21T10:32:45.946 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:45 smithi169 ceph-mon[31469]: pgmap v56: 97 pgs: 97 active+clean; 49 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 342 op/s 2024-04-21T10:32:46.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.051+0000 7fab1465c640 1 -- 172.21.15.26:0/1854940791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab0c102b40 msgr2=0x7fab0c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.051+0000 7fab1465c640 1 --2- 172.21.15.26:0/1854940791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab0c102b40 0x7fab0c106b80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7faafc009900 tx=0x7faafc02bc30 comp rx=0 tx=0).stop 2024-04-21T10:32:46.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.051+0000 7fab1465c640 1 -- 172.21.15.26:0/1854940791 shutdown_connections 2024-04-21T10:32:46.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.051+0000 7fab1465c640 1 --2- 172.21.15.26:0/1854940791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab0c107260 0x7fab0c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.051+0000 7fab1465c640 1 --2- 172.21.15.26:0/1854940791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab0c102b40 0x7fab0c106b80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.051+0000 7fab1465c640 1 --2- 172.21.15.26:0/1854940791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab0c102190 0x7fab0c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.051+0000 7fab1465c640 1 -- 172.21.15.26:0/1854940791 >> 172.21.15.26:0/1854940791 conn(0x7fab0c0793e0 msgr2=0x7fab0c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.051+0000 7fab1465c640 1 -- 172.21.15.26:0/1854940791 shutdown_connections 2024-04-21T10:32:46.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.051+0000 7fab1465c640 1 -- 172.21.15.26:0/1854940791 wait complete. 2024-04-21T10:32:46.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab1465c640 1 Processor -- start 2024-04-21T10:32:46.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab1465c640 1 -- start start 2024-04-21T10:32:46.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab1465c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab0c102190 0x7fab0c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab1465c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab0c107260 0x7fab0c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab1465c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab0c19d000 0x7fab0c19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab1465c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1121a0 con 0x7fab0c102190 2024-04-21T10:32:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab1465c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c112020 con 0x7fab0c107260 2024-04-21T10:32:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab1465c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c112320 con 0x7fab0c19d000 2024-04-21T10:32:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab123f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab0c102190 0x7fab0c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab123f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab0c102190 0x7fab0c198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52286/0 (socket says 172.21.15.26:52286) 2024-04-21T10:32:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab123f8640 1 -- 172.21.15.26:0/1604229418 learned_addr learned my addr 172.21.15.26:0/1604229418 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab12bf9640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab0c19d000 0x7fab0c19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab11bf7640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab0c107260 0x7fab0c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab123f8640 1 -- 172.21.15.26:0/1604229418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab0c19d000 msgr2=0x7fab0c19d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab123f8640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab0c19d000 0x7fab0c19d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab123f8640 1 -- 172.21.15.26:0/1604229418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab0c107260 msgr2=0x7fab0c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab123f8640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab0c107260 0x7fab0c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.052+0000 7fab123f8640 1 -- 172.21.15.26:0/1604229418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c19daf0 con 0x7fab0c102190 2024-04-21T10:32:46.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.053+0000 7fab123f8640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab0c102190 0x7fab0c198740 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fab0800d770 tx=0x7fab0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.053+0000 7fab037fe640 1 -- 172.21.15.26:0/1604229418 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08017070 con 0x7fab0c102190 2024-04-21T10:32:46.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.053+0000 7fab1465c640 1 -- 172.21.15.26:0/1604229418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c19dde0 con 0x7fab0c102190 2024-04-21T10:32:46.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.053+0000 7fab037fe640 1 -- 172.21.15.26:0/1604229418 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab080054d0 con 0x7fab0c102190 2024-04-21T10:32:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.053+0000 7fab037fe640 1 -- 172.21.15.26:0/1604229418 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08004c30 con 0x7fab0c102190 2024-04-21T10:32:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.053+0000 7fab1465c640 1 -- 172.21.15.26:0/1604229418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c1a5580 con 0x7fab0c102190 2024-04-21T10:32:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.054+0000 7fab1465c640 1 -- 172.21.15.26:0/1604229418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c069230 con 0x7fab0c102190 2024-04-21T10:32:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.055+0000 7fab037fe640 1 -- 172.21.15.26:0/1604229418 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fab08004e10 con 0x7fab0c102190 2024-04-21T10:32:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.055+0000 7fab037fe640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7faaf80a3d20 0x7faaf80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.056+0000 7fab11bf7640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7faaf80a3d20 0x7faaf80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.056+0000 7fab037fe640 1 -- 172.21.15.26:0/1604229418 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7fab080d0370 con 0x7fab0c102190 2024-04-21T10:32:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.056+0000 7fab11bf7640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7faaf80a3d20 0x7faaf80a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faafc036040 tx=0x7faafc000bc0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.060+0000 7fab037fe640 1 -- 172.21.15.26:0/1604229418 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab0809ca20 con 0x7fab0c102190 2024-04-21T10:32:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.289+0000 7fab1465c640 1 -- 172.21.15.26:0/1604229418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab0c10a490 con 0x7fab0c102190 2024-04-21T10:32:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.289+0000 7fab037fe640 1 -- 172.21.15.26:0/1604229418 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11890 (secure 0 0 0) 0x7fab0809c320 con 0x7fab0c102190 2024-04-21T10:32:46.289 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:32:46.289 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:32:12.493893+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24478,24488]},"id":1}]} 2024-04-21T10:32:46.290 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 24 2024-04-21T10:32:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.291+0000 7fab1465c640 1 -- 172.21.15.26:0/1604229418 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7faaf80a3d20 msgr2=0x7faaf80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.291+0000 7fab1465c640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7faaf80a3d20 0x7faaf80a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faafc036040 tx=0x7faafc000bc0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.291+0000 7fab1465c640 1 -- 172.21.15.26:0/1604229418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab0c102190 msgr2=0x7fab0c198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.291+0000 7fab1465c640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab0c102190 0x7fab0c198740 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fab0800d770 tx=0x7fab0800dc30 comp rx=0 tx=0).stop 2024-04-21T10:32:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.291+0000 7fab1465c640 1 -- 172.21.15.26:0/1604229418 shutdown_connections 2024-04-21T10:32:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.291+0000 7fab1465c640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7faaf80a3d20 0x7faaf80a61e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.291+0000 7fab1465c640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab0c19d000 0x7fab0c19d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.291+0000 7fab1465c640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab0c107260 0x7fab0c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.291+0000 7fab1465c640 1 --2- 172.21.15.26:0/1604229418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab0c102190 0x7fab0c198740 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.291+0000 7fab1465c640 1 -- 172.21.15.26:0/1604229418 >> 172.21.15.26:0/1604229418 conn(0x7fab0c0793e0 msgr2=0x7fab0c0fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.292+0000 7fab1465c640 1 -- 172.21.15.26:0/1604229418 shutdown_connections 2024-04-21T10:32:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.292+0000 7fab1465c640 1 -- 172.21.15.26:0/1604229418 wait complete. 2024-04-21T10:32:46.303 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24496 quiesce db --cancel --all' 2024-04-21T10:32:46.303 DEBUG:teuthology.orchestra.run.smithi026:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24496 quiesce db --cancel --all 2024-04-21T10:32:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.371+0000 7f4293352640 1 -- 172.21.15.26:0/3287748485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c107280 msgr2=0x7f428c10ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.371+0000 7f4293352640 1 --2- 172.21.15.26:0/3287748485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c107280 0x7f428c10ee50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4284009ca0 tx=0x7f428402c0f0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.371+0000 7f4293352640 1 -- 172.21.15.26:0/3287748485 shutdown_connections 2024-04-21T10:32:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.371+0000 7f4293352640 1 --2- 172.21.15.26:0/3287748485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c107280 0x7f428c10ee50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.371+0000 7f4293352640 1 --2- 172.21.15.26:0/3287748485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f428c102b60 0x7f428c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.371+0000 7f4293352640 1 --2- 172.21.15.26:0/3287748485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f428c1021b0 0x7f428c102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.371+0000 7f4293352640 1 -- 172.21.15.26:0/3287748485 >> 172.21.15.26:0/3287748485 conn(0x7f428c0fda10 msgr2=0x7f428c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.371+0000 7f4293352640 1 -- 172.21.15.26:0/3287748485 shutdown_connections 2024-04-21T10:32:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.371+0000 7f4293352640 1 -- 172.21.15.26:0/3287748485 wait complete. 2024-04-21T10:32:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.372+0000 7f4293352640 1 Processor -- start 2024-04-21T10:32:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.372+0000 7f4293352640 1 -- start start 2024-04-21T10:32:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.372+0000 7f4293352640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f428c1021b0 0x7f428c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.372+0000 7f4293352640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f428c107280 0x7f428c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.372+0000 7f4293352640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c074cd0 0x7f428c1a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.372+0000 7f4293352640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1121d0 con 0x7f428c074cd0 2024-04-21T10:32:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.372+0000 7f4293352640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f428c112050 con 0x7f428c107280 2024-04-21T10:32:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.372+0000 7f4293352640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f428c112350 con 0x7f428c1021b0 2024-04-21T10:32:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.372+0000 7f42918ef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c074cd0 0x7f428c1a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.372+0000 7f42918ef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c074cd0 0x7f428c1a6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52298/0 (socket says 172.21.15.26:52298) 2024-04-21T10:32:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.372+0000 7f42918ef640 1 -- 172.21.15.26:0/606097365 learned_addr learned my addr 172.21.15.26:0/606097365 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.372+0000 7f42910ee640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f428c1021b0 0x7f428c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.373+0000 7f42908ed640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f428c107280 0x7f428c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.373+0000 7f42918ef640 1 -- 172.21.15.26:0/606097365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f428c1021b0 msgr2=0x7f428c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.373+0000 7f42918ef640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f428c1021b0 0x7f428c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.373+0000 7f42918ef640 1 -- 172.21.15.26:0/606097365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f428c107280 msgr2=0x7f428c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.373+0000 7f42918ef640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f428c107280 0x7f428c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.373+0000 7f42918ef640 1 -- 172.21.15.26:0/606097365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c1a6ea0 con 0x7f428c074cd0 2024-04-21T10:32:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.373+0000 7f42918ef640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c074cd0 0x7f428c1a6800 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4284005e30 tx=0x7f4284032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.373+0000 7f42827fc640 1 -- 172.21.15.26:0/606097365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284041020 con 0x7f428c074cd0 2024-04-21T10:32:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.373+0000 7f4293352640 1 -- 172.21.15.26:0/606097365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c1a7130 con 0x7f428c074cd0 2024-04-21T10:32:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.373+0000 7f42827fc640 1 -- 172.21.15.26:0/606097365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4284032970 con 0x7f428c074cd0 2024-04-21T10:32:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.373+0000 7f42827fc640 1 -- 172.21.15.26:0/606097365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284039040 con 0x7f428c074cd0 2024-04-21T10:32:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.373+0000 7f4293352640 1 -- 172.21.15.26:0/606097365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c1a7620 con 0x7f428c074cd0 2024-04-21T10:32:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.376+0000 7f42827fc640 1 -- 172.21.15.26:0/606097365 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4284044020 con 0x7f428c074cd0 2024-04-21T10:32:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.376+0000 7f4296025740 1 Processor -- start 2024-04-21T10:32:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.376+0000 7f4296025740 1 -- start start 2024-04-21T10:32:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.376+0000 7f4296025740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a018d0 0x559ac3a00620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.376+0000 7f4296025740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 0x559ac3a00fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.376+0000 7f4296025740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a0a0e0 0x559ac3a0c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.376+0000 7f4296025740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559ac39658d0 con 0x559ac3a00b60 2024-04-21T10:32:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.376+0000 7f4296025740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559ac3902e80 con 0x559ac3a0a0e0 2024-04-21T10:32:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.376+0000 7f4296025740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559ac3961890 con 0x559ac3a018d0 2024-04-21T10:32:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.376+0000 7f42827fc640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f42780a3dc0 0x7f42780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.376+0000 7f42910ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 0x559ac3a00fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42910ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 0x559ac3a00fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52304/0 (socket says 172.21.15.26:52304) 2024-04-21T10:32:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42908ed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a018d0 0x559ac3a00620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42910ee640 1 -- 172.21.15.26:0/809131836 learned_addr learned my addr 172.21.15.26:0/809131836 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42918ef640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f42780a3dc0 0x7f42780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42908ed640 1 --2- 172.21.15.26:0/809131836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a0a0e0 0x559ac3a0c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42910ee640 1 -- 172.21.15.26:0/809131836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a018d0 msgr2=0x559ac3a00620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42910ee640 1 --2- 172.21.15.26:0/809131836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a018d0 0x559ac3a00620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42910ee640 1 -- 172.21.15.26:0/809131836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a0a0e0 msgr2=0x559ac3a0c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42910ee640 1 --2- 172.21.15.26:0/809131836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a0a0e0 0x559ac3a0c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42827fc640 1 -- 172.21.15.26:0/606097365 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f42840f23f0 con 0x7f428c074cd0 2024-04-21T10:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42910ee640 1 -- 172.21.15.26:0/809131836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ac3924320 con 0x559ac3a00b60 2024-04-21T10:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42908ed640 1 --2- 172.21.15.26:0/809131836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a018d0 0x559ac3a00620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42910ee640 1 --2- 172.21.15.26:0/809131836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 0x559ac3a00fe0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4288009cf0 tx=0x7f4288011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9882ff591f5ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42918ef640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f42780a3dc0 0x7f42780a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42840ba910 tx=0x7f4284090fa0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42777fe640 1 -- 172.21.15.26:0/809131836 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288004230 con 0x559ac3a00b60 2024-04-21T10:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42777fe640 1 -- 172.21.15.26:0/809131836 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42880043d0 con 0x559ac3a00b60 2024-04-21T10:32:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.377+0000 7f42777fe640 1 -- 172.21.15.26:0/809131836 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42880197c0 con 0x559ac3a00b60 2024-04-21T10:32:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.378+0000 7f4296025740 1 -- 172.21.15.26:0/809131836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 msgr2=0x559ac3a00fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.378+0000 7f4296025740 1 --2- 172.21.15.26:0/809131836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 0x559ac3a00fe0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4288009cf0 tx=0x7f4288011810 comp rx=0 tx=0).stop 2024-04-21T10:32:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.378+0000 7f4296025740 1 -- 172.21.15.26:0/809131836 shutdown_connections 2024-04-21T10:32:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.378+0000 7f4296025740 1 --2- 172.21.15.26:0/809131836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a0a0e0 0x559ac3a0c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.378+0000 7f4296025740 1 --2- 172.21.15.26:0/809131836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 0x559ac3a00fe0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.378+0000 7f4296025740 1 --2- 172.21.15.26:0/809131836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a018d0 0x559ac3a00620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.378+0000 7f4296025740 1 -- 172.21.15.26:0/809131836 >> 172.21.15.26:0/809131836 conn(0x559ac39c7c90 msgr2=0x559ac39fb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.378+0000 7f4296025740 1 -- 172.21.15.26:0/809131836 shutdown_connections 2024-04-21T10:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.378+0000 7f4296025740 1 -- 172.21.15.26:0/809131836 wait complete. 2024-04-21T10:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.378+0000 7f4296025740 1 -- start start 2024-04-21T10:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.378+0000 7f4296025740 1 Processor -- start 2024-04-21T10:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f4296025740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 0x559ac3a09a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f4296025740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a018d0 0x559ac3a06df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f4296025740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a0a0e0 0x559ac3a07370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f4296025740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559ac3a0ce10 con 0x559ac3a00b60 2024-04-21T10:32:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f4296025740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559ac38edeb0 con 0x559ac3a0a0e0 2024-04-21T10:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f4296025740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x559ac3a0cf90 con 0x559ac3a018d0 2024-04-21T10:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f4296025740 10 client.0 _ll_register_callbacks cb 0x559ac391ab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f42908ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 0x559ac3a09a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f42908ed640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 0x559ac3a09a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52308/0 (socket says 172.21.15.26:52308) 2024-04-21T10:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f42908ed640 1 -- 172.21.15.26:0/1800860219 learned_addr learned my addr 172.21.15.26:0/1800860219 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f42910ee640 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a018d0 0x559ac3a06df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f42908ed640 1 -- 172.21.15.26:0/1800860219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a018d0 msgr2=0x559ac3a06df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f42908ed640 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a018d0 0x559ac3a06df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f42908ed640 1 -- 172.21.15.26:0/1800860219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a0a0e0 msgr2=0x559ac3a07370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f42908ed640 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a0a0e0 0x559ac3a07370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f42908ed640 1 -- 172.21.15.26:0/1800860219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ac3678210 con 0x559ac3a00b60 2024-04-21T10:32:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.379+0000 7f42910ee640 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a018d0 0x559ac3a06df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:32:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f42908ed640 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 0x559ac3a09a60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f427c005930 tx=0x7f427c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65454117b6589b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f4274ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:32:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f4274ff9640 1 -- 172.21.15.26:0/1800860219 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c01e5f0 con 0x559ac3a00b60 2024-04-21T10:32:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f4274ff9640 1 -- 172.21.15.26:0/1800860219 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f427c01e790 con 0x559ac3a00b60 2024-04-21T10:32:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f4296025740 1 -- 172.21.15.26:0/1800860219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42880043d0 con 0x559ac3a00b60 2024-04-21T10:32:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f4274ff9640 1 -- 172.21.15.26:0/1800860219 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f427c01e990 con 0x559ac3a00b60 2024-04-21T10:32:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f4296025740 10 client.15804 fetch_fsmap learned FSMap version 24 2024-04-21T10:32:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f4296025740 1 -- 172.21.15.26:0/1800860219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559ac3744620 con 0x559ac3a00b60 2024-04-21T10:32:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f4274ff9640 1 -- 172.21.15.26:0/1800860219 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42880043d0 con 0x559ac3a00b60 2024-04-21T10:32:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f4274ff9640 1 -- 172.21.15.26:0/1800860219 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f427c00ec90 con 0x559ac3a00b60 2024-04-21T10:32:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f4296025740 10 client.15804 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T10:32:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f4296025740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:32:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.380+0000 7f4296025740 20 client.15804 populate_metadata read hostname 'smithi026' 2024-04-21T10:32:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.381+0000 7f4296025740 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x559ac3a1a140 0x559ac3a1c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.381+0000 7f4296025740 4 client.15804 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:32:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.381+0000 7f4296025740 1 -- 172.21.15.26:0/1800860219 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ac3979880 con 0x559ac3a1a140 2024-04-21T10:32:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.381+0000 7f42910ee640 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x559ac3a1a140 0x559ac3a1c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.381+0000 7f4274ff9640 10 client.15804 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.381+0000 7f42910ee640 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x559ac3a1a140 0x559ac3a1c570 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b231a6e06ac1e25e server_cookie=abfe22a64333b398 in_seq=0 out_seq=0 2024-04-21T10:32:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4274ff9640 1 -- 172.21.15.26:0/1800860219 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ac3979880 con 0x559ac3a1a140 2024-04-21T10:32:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4274ff9640 10 client.15804 handle_command_reply: tid=0 2024-04-21T10:32:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4296025740 1 client.15804 shutdown 2024-04-21T10:32:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4296025740 1 -- 172.21.15.26:0/1800860219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 msgr2=0x559ac3a09a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4296025740 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 0x559ac3a09a60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f427c005930 tx=0x7f427c00f7f0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4296025740 1 -- 172.21.15.26:0/1800860219 shutdown_connections 2024-04-21T10:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4296025740 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x559ac3a1a140 0x559ac3a1c570 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4296025740 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a0a0e0 0x559ac3a07370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4296025740 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a018d0 0x559ac3a06df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4274ff9640 0 client.15804 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4296025740 1 --2- 172.21.15.26:0/1800860219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a00b60 0x559ac3a09a60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4296025740 1 -- 172.21.15.26:0/1800860219 >> 172.21.15.26:0/1800860219 conn(0x559ac39c7c90 msgr2=0x559ac39fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4296025740 1 -- 172.21.15.26:0/1800860219 shutdown_connections 2024-04-21T10:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.382+0000 7f4296025740 1 -- 172.21.15.26:0/1800860219 wait complete. 2024-04-21T10:32:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.383+0000 7f4296025740 20 client.15804 trim_cache size 0 max 16384 2024-04-21T10:32:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f4296025740 1 Processor -- start 2024-04-21T10:32:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f4296025740 1 -- start start 2024-04-21T10:32:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f4296025740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a00b60 0x559ac3a00080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f4296025740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 0x559ac3a15d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f4296025740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a0a0e0 0x559ac3a09520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f4296025740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1b2600 con 0x559ac3a018d0 2024-04-21T10:32:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f4296025740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1afe70 con 0x559ac3a00b60 2024-04-21T10:32:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f4296025740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1adcc0 con 0x559ac3a0a0e0 2024-04-21T10:32:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f42910ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 0x559ac3a15d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f42910ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 0x559ac3a15d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52312/0 (socket says 172.21.15.26:52312) 2024-04-21T10:32:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f42910ee640 1 -- 172.21.15.26:0/3161270241 learned_addr learned my addr 172.21.15.26:0/3161270241 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f42908ed640 1 --2- 172.21.15.26:0/3161270241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a00b60 0x559ac3a00080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f42908ed640 1 --2- 172.21.15.26:0/3161270241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a0a0e0 0x559ac3a09520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f42910ee640 1 -- 172.21.15.26:0/3161270241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a0a0e0 msgr2=0x559ac3a09520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f42910ee640 1 --2- 172.21.15.26:0/3161270241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a0a0e0 0x559ac3a09520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f42910ee640 1 -- 172.21.15.26:0/3161270241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a00b60 msgr2=0x559ac3a00080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f42910ee640 1 --2- 172.21.15.26:0/3161270241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a00b60 0x559ac3a00080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f42910ee640 1 -- 172.21.15.26:0/3161270241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ac39ffe30 con 0x559ac3a018d0 2024-04-21T10:32:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f42908ed640 1 --2- 172.21.15.26:0/3161270241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a0a0e0 0x559ac3a09520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:32:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.398+0000 7f42910ee640 1 --2- 172.21.15.26:0/3161270241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 0x559ac3a15d10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4288011d10 tx=0x7f4288005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128056168526baa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.399+0000 7f4276ffd640 1 -- 172.21.15.26:0/3161270241 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428802cd50 con 0x559ac3a018d0 2024-04-21T10:32:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.399+0000 7f4276ffd640 1 -- 172.21.15.26:0/3161270241 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4288003aa0 con 0x559ac3a018d0 2024-04-21T10:32:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.399+0000 7f4276ffd640 1 -- 172.21.15.26:0/3161270241 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428802c2a0 con 0x559ac3a018d0 2024-04-21T10:32:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.399+0000 7f4296025740 1 -- 172.21.15.26:0/3161270241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 msgr2=0x559ac3a15d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.399+0000 7f4296025740 1 --2- 172.21.15.26:0/3161270241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 0x559ac3a15d10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4288011d10 tx=0x7f4288005850 comp rx=0 tx=0).stop 2024-04-21T10:32:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.399+0000 7f4296025740 1 -- 172.21.15.26:0/3161270241 shutdown_connections 2024-04-21T10:32:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.399+0000 7f4296025740 1 --2- 172.21.15.26:0/3161270241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a0a0e0 0x559ac3a09520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.399+0000 7f4296025740 1 --2- 172.21.15.26:0/3161270241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 0x559ac3a15d10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.399+0000 7f4296025740 1 --2- 172.21.15.26:0/3161270241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a00b60 0x559ac3a00080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.399+0000 7f4296025740 1 -- 172.21.15.26:0/3161270241 >> 172.21.15.26:0/3161270241 conn(0x559ac39c7c90 msgr2=0x559ac39fb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f4296025740 1 -- 172.21.15.26:0/3161270241 shutdown_connections 2024-04-21T10:32:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f4296025740 1 -- 172.21.15.26:0/3161270241 wait complete. 2024-04-21T10:32:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f4296025740 1 -- start start 2024-04-21T10:32:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f4296025740 1 Processor -- start 2024-04-21T10:32:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f4296025740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a00b60 0x559ac3a12220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f4296025740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 0x559ac3a12760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f4296025740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a0a0e0 0x559ac3a04260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f4296025740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1ae330 con 0x559ac3a018d0 2024-04-21T10:32:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f4296025740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1af7a0 con 0x559ac3a00b60 2024-04-21T10:32:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f4296025740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1aea00 con 0x559ac3a0a0e0 2024-04-21T10:32:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f4296025740 10 client.0 _ll_register_callbacks cb 0x7f427c012aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:32:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f42910ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 0x559ac3a12760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f42910ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 0x559ac3a12760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52320/0 (socket says 172.21.15.26:52320) 2024-04-21T10:32:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f42910ee640 1 -- 172.21.15.26:0/4142735345 learned_addr learned my addr 172.21.15.26:0/4142735345 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.400+0000 7f42908ed640 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a0a0e0 0x559ac3a04260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f42908ed640 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a00b60 0x559ac3a12220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f42910ee640 1 -- 172.21.15.26:0/4142735345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a0a0e0 msgr2=0x559ac3a04260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f42910ee640 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a0a0e0 0x559ac3a04260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f42910ee640 1 -- 172.21.15.26:0/4142735345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a00b60 msgr2=0x559ac3a12220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f42910ee640 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a00b60 0x559ac3a12220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f42910ee640 1 -- 172.21.15.26:0/4142735345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ac3a04a90 con 0x559ac3a018d0 2024-04-21T10:32:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f42908ed640 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a0a0e0 0x559ac3a04260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:32:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f42910ee640 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 0x559ac3a12760 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4288005fd0 tx=0x7f4288011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29e86fe20ecc74 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f4274ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:32:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f4274ff9640 1 -- 172.21.15.26:0/4142735345 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428803a060 con 0x559ac3a018d0 2024-04-21T10:32:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f4274ff9640 1 -- 172.21.15.26:0/4142735345 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f4288005070 con 0x559ac3a018d0 2024-04-21T10:32:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f4296025740 1 -- 172.21.15.26:0/4142735345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4288003aa0 con 0x559ac3a018d0 2024-04-21T10:32:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f4274ff9640 1 -- 172.21.15.26:0/4142735345 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4288005290 con 0x559ac3a018d0 2024-04-21T10:32:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f4296025740 10 client.15810 fetch_fsmap learned FSMap version 24 2024-04-21T10:32:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f4296025740 1 -- 172.21.15.26:0/4142735345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559ac3a0b210 con 0x559ac3a018d0 2024-04-21T10:32:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.401+0000 7f4274ff9640 1 -- 172.21.15.26:0/4142735345 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4288003aa0 con 0x559ac3a018d0 2024-04-21T10:32:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.402+0000 7f4274ff9640 1 -- 172.21.15.26:0/4142735345 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f428803ec40 con 0x559ac3a018d0 2024-04-21T10:32:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.402+0000 7f4296025740 10 client.15810 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T10:32:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.402+0000 7f4296025740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-21T10:32:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.402+0000 7f4296025740 20 client.15810 populate_metadata read hostname 'smithi026' 2024-04-21T10:32:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.402+0000 7f4296025740 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x559ac3a808a0 0x559ac3a82cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.402+0000 7f4296025740 4 client.15810 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T10:32:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.402+0000 7f4296025740 1 -- 172.21.15.26:0/4142735345 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ac39053d0 con 0x559ac3a808a0 2024-04-21T10:32:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.402+0000 7f42908ed640 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x559ac3a808a0 0x559ac3a82cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.402+0000 7f4274ff9640 10 client.15810 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.402+0000 7f42908ed640 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x559ac3a808a0 0x559ac3a82cc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21bcd127df9c51d2 server_cookie=d33101aaa00823e7 in_seq=0 out_seq=0 2024-04-21T10:32:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.418+0000 7f31313bd640 1 -- 172.21.15.26:0/1092391504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c1010a0 msgr2=0x7f312c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.418+0000 7f31313bd640 1 --2- 172.21.15.26:0/1092391504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c1010a0 0x7f312c101480 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f311c00bc50 tx=0x7f311c02db60 comp rx=0 tx=0).stop 2024-04-21T10:32:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.418+0000 7f31313bd640 1 -- 172.21.15.26:0/1092391504 shutdown_connections 2024-04-21T10:32:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.418+0000 7f31313bd640 1 --2- 172.21.15.26:0/1092391504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f312c106170 0x7f312c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.418+0000 7f31313bd640 1 --2- 172.21.15.26:0/1092391504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c101a50 0x7f312c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.418+0000 7f31313bd640 1 --2- 172.21.15.26:0/1092391504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c1010a0 0x7f312c101480 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.418+0000 7f31313bd640 1 -- 172.21.15.26:0/1092391504 >> 172.21.15.26:0/1092391504 conn(0x7f312c077eb0 msgr2=0x7f312c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.418+0000 7f31313bd640 1 -- 172.21.15.26:0/1092391504 shutdown_connections 2024-04-21T10:32:46.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.418+0000 7f31313bd640 1 -- 172.21.15.26:0/1092391504 wait complete. 2024-04-21T10:32:46.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.419+0000 7f31313bd640 1 Processor -- start 2024-04-21T10:32:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.420+0000 7f31313bd640 1 -- start start 2024-04-21T10:32:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.420+0000 7f31313bd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f312c1010a0 0x7f312c10c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.420+0000 7f31313bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c106170 0x7f312c10ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.420+0000 7f31313bd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c10d340 0x7f312c1b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.420+0000 7f31313bd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f312c10fb90 con 0x7f312c106170 2024-04-21T10:32:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.420+0000 7f31313bd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f312c10fa10 con 0x7f312c10d340 2024-04-21T10:32:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.420+0000 7f31313bd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f312c10fd10 con 0x7f312c1010a0 2024-04-21T10:32:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.420+0000 7f312a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c106170 0x7f312c10ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.420+0000 7f312a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c106170 0x7f312c10ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52338/0 (socket says 172.21.15.26:52338) 2024-04-21T10:32:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.420+0000 7f312a7fc640 1 -- 172.21.15.26:0/3789368947 learned_addr learned my addr 172.21.15.26:0/3789368947 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.421+0000 7f312b7fe640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c10d340 0x7f312c1b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.421+0000 7f312affd640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f312c1010a0 0x7f312c10c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.421+0000 7f312a7fc640 1 -- 172.21.15.26:0/3789368947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f312c1010a0 msgr2=0x7f312c10c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.421+0000 7f312a7fc640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f312c1010a0 0x7f312c10c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.421+0000 7f312a7fc640 1 -- 172.21.15.26:0/3789368947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c10d340 msgr2=0x7f312c1b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.421+0000 7f312a7fc640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c10d340 0x7f312c1b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.421+0000 7f312a7fc640 1 -- 172.21.15.26:0/3789368947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c1b3bd0 con 0x7f312c106170 2024-04-21T10:32:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.421+0000 7f312b7fe640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c10d340 0x7f312c1b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:32:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.422+0000 7f312a7fc640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c106170 0x7f312c10ce00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f312400eda0 tx=0x7f312400c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.423+0000 7f3113fff640 1 -- 172.21.15.26:0/3789368947 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312401c070 con 0x7f312c106170 2024-04-21T10:32:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.423+0000 7f31313bd640 1 -- 172.21.15.26:0/3789368947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c1b3e60 con 0x7f312c106170 2024-04-21T10:32:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.423+0000 7f31313bd640 1 -- 172.21.15.26:0/3789368947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c1b4320 con 0x7f312c106170 2024-04-21T10:32:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.423+0000 7f3113fff640 1 -- 172.21.15.26:0/3789368947 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3124017af0 con 0x7f312c106170 2024-04-21T10:32:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.423+0000 7f3113fff640 1 -- 172.21.15.26:0/3789368947 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312400fc50 con 0x7f312c106170 2024-04-21T10:32:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f3113fff640 1 -- 172.21.15.26:0/3789368947 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f312401f070 con 0x7f312c106170 2024-04-21T10:32:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f3133fb5740 1 Processor -- start 2024-04-21T10:32:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f3133fb5740 1 -- start start 2024-04-21T10:32:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f3133fb5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 0x55c7c82b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f3133fb5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82bdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f3133fb5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82c0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f3133fb5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c7c821a000 con 0x55c7c82b4d10 2024-04-21T10:32:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f3133fb5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c7c81b6b20 con 0x55c7c82be240 2024-04-21T10:32:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f3133fb5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c7c8215fc0 con 0x55c7c82b56c0 2024-04-21T10:32:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f312b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 0x55c7c82b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f312b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 0x55c7c82b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52340/0 (socket says 172.21.15.26:52340) 2024-04-21T10:32:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f312b7fe640 1 -- 172.21.15.26:0/2065611830 learned_addr learned my addr 172.21.15.26:0/2065611830 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f312affd640 1 --2- 172.21.15.26:0/2065611830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82bdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.426+0000 7f312a7fc640 1 --2- 172.21.15.26:0/2065611830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82c0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f312b7fe640 1 -- 172.21.15.26:0/2065611830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 msgr2=0x55c7c82bdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f312b7fe640 1 --2- 172.21.15.26:0/2065611830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82bdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f312b7fe640 1 -- 172.21.15.26:0/2065611830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 msgr2=0x55c7c82c0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f312b7fe640 1 --2- 172.21.15.26:0/2065611830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82c0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f312b7fe640 1 -- 172.21.15.26:0/2065611830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c81d0ea0 con 0x55c7c82b4d10 2024-04-21T10:32:46.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f312a7fc640 1 --2- 172.21.15.26:0/2065611830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82c0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:32:46.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f3113fff640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f31140a3eb0 0x7f31140a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f312b7fe640 1 --2- 172.21.15.26:0/2065611830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 0x55c7c82b50f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3120010400 tx=0x7f31200108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645b192da0393ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f312affd640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f31140a3eb0 0x7f31140a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f31117fa640 1 -- 172.21.15.26:0/2065611830 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120011760 con 0x55c7c82b4d10 2024-04-21T10:32:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f4274ff9640 1 -- 172.21.15.26:0/4142735345 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ac39053d0 con 0x559ac3a808a0 2024-04-21T10:32:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f4274ff9640 10 client.15810 handle_command_reply: tid=0 2024-04-21T10:32:46.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f3113fff640 1 -- 172.21.15.26:0/3789368947 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f31240d0600 con 0x7f312c106170 2024-04-21T10:32:46.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f31117fa640 1 -- 172.21.15.26:0/2065611830 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3120011db0 con 0x55c7c82b4d10 2024-04-21T10:32:46.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f31117fa640 1 -- 172.21.15.26:0/2065611830 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120018380 con 0x55c7c82b4d10 2024-04-21T10:32:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.427+0000 7f4296025740 1 client.15810 shutdown 2024-04-21T10:32:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f4296025740 1 -- 172.21.15.26:0/4142735345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 msgr2=0x559ac3a12760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f4296025740 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 0x559ac3a12760 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4288005fd0 tx=0x7f4288011dd0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f4296025740 1 -- 172.21.15.26:0/4142735345 shutdown_connections 2024-04-21T10:32:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f4296025740 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x559ac3a808a0 0x559ac3a82cc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f4274ff9640 0 client.15810 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f4296025740 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ac3a0a0e0 0x559ac3a04260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f4296025740 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559ac3a018d0 0x559ac3a12760 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f4296025740 1 --2- 172.21.15.26:0/4142735345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559ac3a00b60 0x559ac3a12220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f4296025740 1 -- 172.21.15.26:0/4142735345 >> 172.21.15.26:0/4142735345 conn(0x559ac39c7c90 msgr2=0x559ac3a05640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f4296025740 1 -- 172.21.15.26:0/4142735345 shutdown_connections 2024-04-21T10:32:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f4296025740 1 -- 172.21.15.26:0/4142735345 wait complete. 2024-04-21T10:32:46.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f3133fb5740 1 -- 172.21.15.26:0/2065611830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 msgr2=0x55c7c82b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f312affd640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f31140a3eb0 0x7f31140a6370 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f311c000c00 tx=0x7f311c037440 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f3133fb5740 1 --2- 172.21.15.26:0/2065611830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 0x55c7c82b50f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3120010400 tx=0x7f31200108c0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f3133fb5740 1 -- 172.21.15.26:0/2065611830 shutdown_connections 2024-04-21T10:32:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f4296025740 20 client.15810 trim_cache size 0 max 16384 2024-04-21T10:32:46.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f3133fb5740 1 --2- 172.21.15.26:0/2065611830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82c0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f3133fb5740 1 --2- 172.21.15.26:0/2065611830 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82bdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f3133fb5740 1 --2- 172.21.15.26:0/2065611830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 0x55c7c82b50f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f3133fb5740 1 -- 172.21.15.26:0/2065611830 >> 172.21.15.26:0/2065611830 conn(0x55c7c82af3e0 msgr2=0x55c7c82af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f3133fb5740 1 -- 172.21.15.26:0/2065611830 shutdown_connections 2024-04-21T10:32:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f3133fb5740 1 -- 172.21.15.26:0/2065611830 wait complete. 2024-04-21T10:32:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:{ 2024-04-21T10:32:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "message": "", 2024-04-21T10:32:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout: "return_code": 0 2024-04-21T10:32:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stdout:} 2024-04-21T10:32:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f3133fb5740 1 -- start start 2024-04-21T10:32:46.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.428+0000 7f3133fb5740 1 Processor -- start 2024-04-21T10:32:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f4293352640 1 -- 172.21.15.26:0/606097365 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f42780a3dc0 msgr2=0x7f42780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f4293352640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f42780a3dc0 0x7f42780a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42840ba910 tx=0x7f4284090fa0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f4293352640 1 -- 172.21.15.26:0/606097365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c074cd0 msgr2=0x7f428c1a6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f4293352640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c074cd0 0x7f428c1a6800 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4284005e30 tx=0x7f4284032420 comp rx=0 tx=0).stop 2024-04-21T10:32:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f4293352640 1 -- 172.21.15.26:0/606097365 shutdown_connections 2024-04-21T10:32:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f4293352640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f42780a3dc0 0x7f42780a6280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f4293352640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c074cd0 0x7f428c1a6800 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f4293352640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f428c107280 0x7f428c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f4293352640 1 --2- 172.21.15.26:0/606097365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f428c1021b0 0x7f428c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f4293352640 1 -- 172.21.15.26:0/606097365 >> 172.21.15.26:0/606097365 conn(0x7f428c0fda10 msgr2=0x7f428c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f3133fb5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82b4d10 0x55c7c82bb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f3133fb5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82bb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f3133fb5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82be240 0x55c7c82bbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f3133fb5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c7c82c10c0 con 0x55c7c82be240 2024-04-21T10:32:46.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f3133fb5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c7c81b4510 con 0x55c7c82b4d10 2024-04-21T10:32:46.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f3133fb5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c7c82c1240 con 0x55c7c82b56c0 2024-04-21T10:32:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f4293352640 1 -- 172.21.15.26:0/606097365 shutdown_connections 2024-04-21T10:32:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi026.front.sepia.ceph.com.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f4293352640 1 -- 172.21.15.26:0/606097365 wait complete. 2024-04-21T10:32:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f3133fb5740 10 client.0 _ll_register_callbacks cb 0x55c7c81b3f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:32:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f312affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82b4d10 0x55c7c82bb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f312b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82be240 0x55c7c82bbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f312a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f312affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82b4d10 0x55c7c82bb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.26:41830/0 (socket says 172.21.15.26:41830) 2024-04-21T10:32:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f312a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82bb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.26:55924/0 (socket says 172.21.15.26:55924) 2024-04-21T10:32:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f312affd640 1 -- 172.21.15.26:0/3776452414 learned_addr learned my addr 172.21.15.26:0/3776452414 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f312a7fc640 1 -- 172.21.15.26:0/3776452414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82b4d10 msgr2=0x55c7c82bb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f312a7fc640 1 --2- 172.21.15.26:0/3776452414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82b4d10 0x55c7c82bb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f312a7fc640 1 -- 172.21.15.26:0/3776452414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82be240 msgr2=0x55c7c82bbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f312a7fc640 1 --2- 172.21.15.26:0/3776452414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82be240 0x55c7c82bbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f312a7fc640 1 -- 172.21.15.26:0/3776452414 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c7ff8a20 con 0x55c7c82b56c0 2024-04-21T10:32:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.429+0000 7f312b7fe640 1 --2- 172.21.15.26:0/3776452414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82be240 0x55c7c82bbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:32:46.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f312a7fc640 1 --2- 172.21.15.26:0/3776452414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82bb610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f31240cde70 tx=0x7f312409c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37168e01f730ce62 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f310eff5640 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2024-04-21T10:32:46.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f310eff5640 1 -- 172.21.15.26:0/3776452414 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240a5330 con 0x55c7c82b56c0 2024-04-21T10:32:46.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f3133fb5740 1 -- 172.21.15.26:0/3776452414 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3120011db0 con 0x55c7c82b56c0 2024-04-21T10:32:46.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f310eff5640 1 -- 172.21.15.26:0/3776452414 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f31240cdfe0 con 0x55c7c82b56c0 2024-04-21T10:32:46.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f310eff5640 1 -- 172.21.15.26:0/3776452414 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3124098de0 con 0x55c7c82b56c0 2024-04-21T10:32:46.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f3133fb5740 10 client.25396 fetch_fsmap learned FSMap version 24 2024-04-21T10:32:46.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f3133fb5740 1 -- 172.21.15.26:0/3776452414 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c7c80591b0 con 0x55c7c82b56c0 2024-04-21T10:32:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f310eff5640 1 -- 172.21.15.26:0/3776452414 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3120011db0 con 0x55c7c82b56c0 2024-04-21T10:32:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f310eff5640 1 -- 172.21.15.26:0/3776452414 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f312400b030 con 0x55c7c82b56c0 2024-04-21T10:32:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f3133fb5740 10 client.25396 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T10:32:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f3133fb5740 10 client.25396 resolve_mds: validated gid 24496 aka daemon mds.i 2024-04-21T10:32:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f3133fb5740 20 client.25396 populate_metadata read hostname 'smithi026' 2024-04-21T10:32:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f3133fb5740 1 --2- 172.21.15.26:0/3776452414 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55c7c82ceb70 0x55c7c82d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f3133fb5740 4 client.25396 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T10:32:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.430+0000 7f3133fb5740 1 -- 172.21.15.26:0/3776452414 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7c8227a20 con 0x55c7c82ceb70 2024-04-21T10:32:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.431+0000 7f312affd640 1 --2- 172.21.15.26:0/3776452414 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55c7c82ceb70 0x55c7c82d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.431+0000 7f310eff5640 10 client.25396 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.431+0000 7f312affd640 1 --2- 172.21.15.26:0/3776452414 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55c7c82ceb70 0x55c7c82d0fb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d22426053e0edb5 server_cookie=d5b4a641b74f40c7 in_seq=0 out_seq=0 2024-04-21T10:32:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f310eff5640 1 -- 172.21.15.26:0/3776452414 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7c8227a20 con 0x55c7c82ceb70 2024-04-21T10:32:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f310eff5640 10 client.25396 handle_command_reply: tid=0 2024-04-21T10:32:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f3133fb5740 1 client.25396 shutdown 2024-04-21T10:32:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f3133fb5740 1 -- 172.21.15.26:0/3776452414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 msgr2=0x55c7c82bb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f3133fb5740 1 --2- 172.21.15.26:0/3776452414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82bb610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f31240cde70 tx=0x7f312409c370 comp rx=0 tx=0).stop 2024-04-21T10:32:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f3133fb5740 1 -- 172.21.15.26:0/3776452414 shutdown_connections 2024-04-21T10:32:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f3133fb5740 1 --2- 172.21.15.26:0/3776452414 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55c7c82ceb70 0x55c7c82d0fb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f3133fb5740 1 --2- 172.21.15.26:0/3776452414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82be240 0x55c7c82bbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f3133fb5740 1 --2- 172.21.15.26:0/3776452414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82bb610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f3133fb5740 1 --2- 172.21.15.26:0/3776452414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82b4d10 0x55c7c82bb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f3133fb5740 1 -- 172.21.15.26:0/3776452414 >> 172.21.15.26:0/3776452414 conn(0x55c7c82af3e0 msgr2=0x55c7c82ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f310eff5640 0 client.25396 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f3133fb5740 1 -- 172.21.15.26:0/3776452414 shutdown_connections 2024-04-21T10:32:46.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f3133fb5740 1 -- 172.21.15.26:0/3776452414 wait complete. 2024-04-21T10:32:46.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.432+0000 7f3133fb5740 20 client.25396 trim_cache size 0 max 16384 2024-04-21T10:32:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.449+0000 7f3133fb5740 1 Processor -- start 2024-04-21T10:32:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.449+0000 7f3133fb5740 1 -- start start 2024-04-21T10:32:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f3133fb5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b4d10 0x55c7c82bd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f3133fb5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b56c0 0x55c7c82ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f3133fb5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82ba030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f3133fb5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3118011070 con 0x55c7c82b56c0 2024-04-21T10:32:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f3133fb5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f311800e8e0 con 0x55c7c82be240 2024-04-21T10:32:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f3133fb5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f311800c730 con 0x55c7c82b4d10 2024-04-21T10:32:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f312a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b56c0 0x55c7c82ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f312a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b56c0 0x55c7c82ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52358/0 (socket says 172.21.15.26:52358) 2024-04-21T10:32:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f312a7fc640 1 -- 172.21.15.26:0/3497594045 learned_addr learned my addr 172.21.15.26:0/3497594045 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f312b7fe640 1 --2- 172.21.15.26:0/3497594045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82ba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f312affd640 1 --2- 172.21.15.26:0/3497594045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b4d10 0x55c7c82bd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f312a7fc640 1 -- 172.21.15.26:0/3497594045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b4d10 msgr2=0x55c7c82bd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f312a7fc640 1 --2- 172.21.15.26:0/3497594045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b4d10 0x55c7c82bd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f312a7fc640 1 -- 172.21.15.26:0/3497594045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 msgr2=0x55c7c82ba030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f312a7fc640 1 --2- 172.21.15.26:0/3497594045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82ba030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f312a7fc640 1 -- 172.21.15.26:0/3497594045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c82b5460 con 0x55c7c82b56c0 2024-04-21T10:32:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.450+0000 7f312a7fc640 1 --2- 172.21.15.26:0/3497594045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b56c0 0x55c7c82ca0a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f312409c700 tx=0x7f312409ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2af6298662a54a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.451+0000 7f3110ff9640 1 -- 172.21.15.26:0/3497594045 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240ba030 con 0x55c7c82b56c0 2024-04-21T10:32:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.451+0000 7f3110ff9640 1 -- 172.21.15.26:0/3497594045 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31240d6030 con 0x55c7c82b56c0 2024-04-21T10:32:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.451+0000 7f3110ff9640 1 -- 172.21.15.26:0/3497594045 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312400c040 con 0x55c7c82b56c0 2024-04-21T10:32:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.451+0000 7f3133fb5740 1 -- 172.21.15.26:0/3497594045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b56c0 msgr2=0x55c7c82ca0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.451+0000 7f3133fb5740 1 --2- 172.21.15.26:0/3497594045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b56c0 0x55c7c82ca0a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f312409c700 tx=0x7f312409ac80 comp rx=0 tx=0).stop 2024-04-21T10:32:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.451+0000 7f3133fb5740 1 -- 172.21.15.26:0/3497594045 shutdown_connections 2024-04-21T10:32:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.451+0000 7f3133fb5740 1 --2- 172.21.15.26:0/3497594045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82ba030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.451+0000 7f3133fb5740 1 --2- 172.21.15.26:0/3497594045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b56c0 0x55c7c82ca0a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.451+0000 7f3133fb5740 1 --2- 172.21.15.26:0/3497594045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b4d10 0x55c7c82bd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.451+0000 7f3133fb5740 1 -- 172.21.15.26:0/3497594045 >> 172.21.15.26:0/3497594045 conn(0x55c7c82af3e0 msgr2=0x55c7c82ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f3133fb5740 1 -- 172.21.15.26:0/3497594045 shutdown_connections 2024-04-21T10:32:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f3133fb5740 1 -- 172.21.15.26:0/3497594045 wait complete. 2024-04-21T10:32:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f3133fb5740 1 -- start start 2024-04-21T10:32:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f3133fb5740 1 Processor -- start 2024-04-21T10:32:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f3133fb5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 0x55c7c82b7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f3133fb5740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82b84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f3133fb5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f3133fb5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f311800cda0 con 0x55c7c82b4d10 2024-04-21T10:32:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f3133fb5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f311800e210 con 0x55c7c82be240 2024-04-21T10:32:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f3133fb5740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f311800d470 con 0x55c7c82b56c0 2024-04-21T10:32:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f3133fb5740 10 client.0 _ll_register_callbacks cb 0x7f3124098aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T10:32:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f312affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 0x55c7c82b7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f312affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 0x55c7c82b7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52364/0 (socket says 172.21.15.26:52364) 2024-04-21T10:32:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f312affd640 1 -- 172.21.15.26:0/1553922008 learned_addr learned my addr 172.21.15.26:0/1553922008 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:32:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f312a7fc640 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82b84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.452+0000 7f312b7fe640 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f312affd640 1 -- 172.21.15.26:0/1553922008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 msgr2=0x55c7c82b84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f312affd640 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82b84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f312affd640 1 -- 172.21.15.26:0/1553922008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 msgr2=0x55c7c82b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T10:32:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f312affd640 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f312affd640 1 -- 172.21.15.26:0/1553922008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c7eec310 con 0x55c7c82b4d10 2024-04-21T10:32:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f312affd640 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 0x55c7c82b7f80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f311c02e170 tx=0x7f311c02db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1796f2228f89ff52 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:32:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f310eff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-21T10:32:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f310eff5640 1 -- 172.21.15.26:0/1553922008 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c037680 con 0x55c7c82b4d10 2024-04-21T10:32:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f310eff5640 1 -- 172.21.15.26:0/1553922008 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8506+0+0 (secure 0 0 0) 0x7f311c037820 con 0x55c7c82b4d10 2024-04-21T10:32:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f3133fb5740 1 -- 172.21.15.26:0/1553922008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31240d6030 con 0x55c7c82b4d10 2024-04-21T10:32:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f310eff5640 1 -- 172.21.15.26:0/1553922008 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f311c061050 con 0x55c7c82b4d10 2024-04-21T10:32:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f3133fb5740 10 client.15828 fetch_fsmap learned FSMap version 24 2024-04-21T10:32:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.453+0000 7f3133fb5740 1 -- 172.21.15.26:0/1553922008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c7c83354a0 con 0x55c7c82b4d10 2024-04-21T10:32:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.456+0000 7f310eff5640 1 -- 172.21.15.26:0/1553922008 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31240d6030 con 0x55c7c82b4d10 2024-04-21T10:32:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.456+0000 7f310eff5640 1 -- 172.21.15.26:0/1553922008 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f311c04d040 con 0x55c7c82b4d10 2024-04-21T10:32:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.456+0000 7f3133fb5740 10 client.15828 fetch_fsmap finished waiting for FSMap version 24 2024-04-21T10:32:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.456+0000 7f3133fb5740 10 client.15828 resolve_mds: validated gid 24496 aka daemon mds.i 2024-04-21T10:32:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.456+0000 7f3133fb5740 20 client.15828 populate_metadata read hostname 'smithi026' 2024-04-21T10:32:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.456+0000 7f3133fb5740 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55c7c8335c50 0x55c7c8338070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:32:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.456+0000 7f3133fb5740 4 client.15828 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-21T10:32:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.456+0000 7f3133fb5740 1 -- 172.21.15.26:0/1553922008 --> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55c7c8221890 con 0x55c7c8335c50 2024-04-21T10:32:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.456+0000 7f312a7fc640 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55c7c8335c50 0x55c7c8338070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:32:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.456+0000 7f310eff5640 10 client.15828 ms_handle_connect on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.456+0000 7f312a7fc640 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55c7c8335c50 0x55c7c8338070 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e592812a54811bf server_cookie=e413cbb90efc4a90 in_seq=0 out_seq=0 2024-04-21T10:32:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f310eff5640 1 -- 172.21.15.26:0/1553922008 <== mds.0 v2:172.21.15.190:6838/1574521545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x55c7c8221890 con 0x55c7c8335c50 2024-04-21T10:32:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f310eff5640 10 client.15828 handle_command_reply: tid=0 2024-04-21T10:32:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f3133fb5740 1 client.15828 shutdown 2024-04-21T10:32:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f3133fb5740 1 -- 172.21.15.26:0/1553922008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 msgr2=0x55c7c82b7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f3133fb5740 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 0x55c7c82b7f80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f311c02e170 tx=0x7f311c02db60 comp rx=0 tx=0).stop 2024-04-21T10:32:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f3133fb5740 1 -- 172.21.15.26:0/1553922008 shutdown_connections 2024-04-21T10:32:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f3133fb5740 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.190:6838/1574521545,v1:172.21.15.190:6839/1574521545] conn(0x55c7c8335c50 0x55c7c8338070 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f3133fb5740 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c7c82be240 0x55c7c82b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f3133fb5740 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7c82b56c0 0x55c7c82b84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f3133fb5740 1 --2- 172.21.15.26:0/1553922008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c7c82b4d10 0x55c7c82b7f80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f3133fb5740 1 -- 172.21.15.26:0/1553922008 >> 172.21.15.26:0/1553922008 conn(0x55c7c82af3e0 msgr2=0x55c7c82addf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f310eff5640 0 client.15828 ms_handle_reset on v2:172.21.15.190:6838/1574521545 2024-04-21T10:32:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f3133fb5740 1 -- 172.21.15.26:0/1553922008 shutdown_connections 2024-04-21T10:32:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f3133fb5740 1 -- 172.21.15.26:0/1553922008 wait complete. 2024-04-21T10:32:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f3133fb5740 20 client.15828 trim_cache size 0 max 16384 2024-04-21T10:32:46.468 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:32:46.468 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 24, 2024-04-21T10:32:46.468 INFO:teuthology.orchestra.run.smithi026.stdout: "leader": 24496, 2024-04-21T10:32:46.468 INFO:teuthology.orchestra.run.smithi026.stdout: "set_version": 8, 2024-04-21T10:32:46.468 INFO:teuthology.orchestra.run.smithi026.stdout: "sets": {} 2024-04-21T10:32:46.469 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:32:46.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f31313bd640 1 -- 172.21.15.26:0/3789368947 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f31140a3eb0 msgr2=0x7f31140a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f31313bd640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f31140a3eb0 0x7f31140a6370 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f311c000c00 tx=0x7f311c037440 comp rx=0 tx=0).stop 2024-04-21T10:32:46.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f31313bd640 1 -- 172.21.15.26:0/3789368947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c106170 msgr2=0x7f312c10ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:32:46.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f31313bd640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c106170 0x7f312c10ce00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f312400eda0 tx=0x7f312400c6e0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f31313bd640 1 -- 172.21.15.26:0/3789368947 shutdown_connections 2024-04-21T10:32:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f31313bd640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f31140a3eb0 0x7f31140a6370 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.457+0000 7f31313bd640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f312c10d340 0x7f312c1b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.458+0000 7f31313bd640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c106170 0x7f312c10ce00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.458+0000 7f31313bd640 1 --2- 172.21.15.26:0/3789368947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f312c1010a0 0x7f312c10c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:32:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.458+0000 7f31313bd640 1 -- 172.21.15.26:0/3789368947 >> 172.21.15.26:0/3789368947 conn(0x7f312c077eb0 msgr2=0x7f312c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:32:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.458+0000 7f31313bd640 1 -- 172.21.15.26:0/3789368947 shutdown_connections 2024-04-21T10:32:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:32:46.458+0000 7f31313bd640 1 -- 172.21.15.26:0/3789368947 wait complete. 2024-04-21T10:32:46.472 INFO:tasks.quiescer:done joining Quiescers 2024-04-21T10:32:46.472 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-21T10:32:46.481 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi026.front.sepia.ceph.com 2024-04-21T10:32:46.489 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-21T10:32:46.499 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-21T10:32:46.499 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T10:32:46.499 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2024-04-21T10:32:46.500 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-21T10:32:46.553 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:46 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1604229418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:32:46.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-21T10:32:46.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:46 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1604229418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:32:46.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:46 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1604229418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:32:47.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:47 smithi190 ceph-mon[31423]: pgmap v57: 97 pgs: 97 active+clean; 50 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 351 op/s 2024-04-21T10:32:47.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:47 smithi026 ceph-mon[27965]: pgmap v57: 97 pgs: 97 active+clean; 50 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 351 op/s 2024-04-21T10:32:47.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:47 smithi169 ceph-mon[31469]: pgmap v57: 97 pgs: 97 active+clean; 50 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 351 op/s 2024-04-21T10:32:49.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:49 smithi190 ceph-mon[31423]: pgmap v58: 97 pgs: 97 active+clean; 50 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 352 KiB/s wr, 54 op/s 2024-04-21T10:32:49.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:49 smithi026 ceph-mon[27965]: pgmap v58: 97 pgs: 97 active+clean; 50 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 352 KiB/s wr, 54 op/s 2024-04-21T10:32:49.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:49 smithi169 ceph-mon[31469]: pgmap v58: 97 pgs: 97 active+clean; 50 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 352 KiB/s wr, 54 op/s 2024-04-21T10:32:51.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:51 smithi190 ceph-mon[31423]: pgmap v59: 97 pgs: 97 active+clean; 50 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 366 KiB/s wr, 73 op/s 2024-04-21T10:32:51.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:51 smithi026 ceph-mon[27965]: pgmap v59: 97 pgs: 97 active+clean; 50 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 366 KiB/s wr, 73 op/s 2024-04-21T10:32:51.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:51 smithi169 ceph-mon[31469]: pgmap v59: 97 pgs: 97 active+clean; 50 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 366 KiB/s wr, 73 op/s 2024-04-21T10:32:53.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:53 smithi190 ceph-mon[31423]: pgmap v60: 97 pgs: 97 active+clean; 50 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 146 KiB/s wr, 27 op/s 2024-04-21T10:32:53.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:53 smithi026 ceph-mon[27965]: pgmap v60: 97 pgs: 97 active+clean; 50 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 146 KiB/s wr, 27 op/s 2024-04-21T10:32:53.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:53 smithi169 ceph-mon[31469]: pgmap v60: 97 pgs: 97 active+clean; 50 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 146 KiB/s wr, 27 op/s 2024-04-21T10:32:55.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:55 smithi190 ceph-mon[31423]: pgmap v61: 97 pgs: 97 active+clean; 50 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 146 KiB/s wr, 27 op/s 2024-04-21T10:32:55.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:55 smithi026 ceph-mon[27965]: pgmap v61: 97 pgs: 97 active+clean; 50 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 146 KiB/s wr, 27 op/s 2024-04-21T10:32:55.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:55 smithi169 ceph-mon[31469]: pgmap v61: 97 pgs: 97 active+clean; 50 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 146 KiB/s wr, 27 op/s 2024-04-21T10:32:57.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:57 smithi190 ceph-mon[31423]: pgmap v62: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.6 MiB/s wr, 249 op/s 2024-04-21T10:32:57.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:57 smithi026 ceph-mon[27965]: pgmap v62: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.6 MiB/s wr, 249 op/s 2024-04-21T10:32:57.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:57 smithi169 ceph-mon[31469]: pgmap v62: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.6 MiB/s wr, 249 op/s 2024-04-21T10:32:59.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:32:59 smithi190 ceph-mon[31423]: pgmap v63: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-04-21T10:32:59.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:32:59 smithi026 ceph-mon[27965]: pgmap v63: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-04-21T10:32:59.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:32:59 smithi169 ceph-mon[31469]: pgmap v63: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-04-21T10:33:01.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:01 smithi190 ceph-mon[31423]: pgmap v64: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.5 MiB/s wr, 245 op/s 2024-04-21T10:33:01.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:01 smithi026 ceph-mon[27965]: pgmap v64: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.5 MiB/s wr, 245 op/s 2024-04-21T10:33:01.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:01 smithi169 ceph-mon[31469]: pgmap v64: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.5 MiB/s wr, 245 op/s 2024-04-21T10:33:03.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:03 smithi190 ceph-mon[31423]: pgmap v65: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-04-21T10:33:03.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:03 smithi026 ceph-mon[27965]: pgmap v65: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-04-21T10:33:03.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:03 smithi169 ceph-mon[31469]: pgmap v65: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-04-21T10:33:04.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:04 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:33:04.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:04 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:33:04.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:04 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:33:05.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:05 smithi190 ceph-mon[31423]: pgmap v66: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-04-21T10:33:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:05 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:05 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:05 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:05 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:05 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:33:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:05 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:33:05.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:05 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.836 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:05 smithi026 ceph-mon[27965]: pgmap v66: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-04-21T10:33:05.837 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:05 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.837 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:05 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.837 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:05 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.837 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:05 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.837 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:05 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:33:05.837 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:05 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:33:05.837 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:05 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:05 smithi169 ceph-mon[31469]: pgmap v66: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-04-21T10:33:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:05 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:05 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:05 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:05 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:05 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:33:05.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:05 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:33:05.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:05 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:33:07.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:07 smithi190 ceph-mon[31423]: pgmap v67: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.5 MiB/s wr, 244 op/s 2024-04-21T10:33:07.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:07 smithi026 ceph-mon[27965]: pgmap v67: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.5 MiB/s wr, 244 op/s 2024-04-21T10:33:07.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:07 smithi169 ceph-mon[31469]: pgmap v67: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.5 MiB/s wr, 244 op/s 2024-04-21T10:33:09.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:09 smithi190 ceph-mon[31423]: pgmap v68: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-21T10:33:09.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:09 smithi026 ceph-mon[27965]: pgmap v68: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-21T10:33:09.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:09 smithi169 ceph-mon[31469]: pgmap v68: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-21T10:33:11.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:11 smithi190 ceph-mon[31423]: pgmap v69: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 26 op/s 2024-04-21T10:33:11.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:11 smithi026 ceph-mon[27965]: pgmap v69: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 26 op/s 2024-04-21T10:33:11.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:11 smithi169 ceph-mon[31469]: pgmap v69: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 26 op/s 2024-04-21T10:33:13.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:13 smithi190 ceph-mon[31423]: pgmap v70: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-04-21T10:33:13.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:13 smithi026 ceph-mon[27965]: pgmap v70: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-04-21T10:33:13.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:13 smithi169 ceph-mon[31469]: pgmap v70: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-04-21T10:33:15.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:15 smithi190 ceph-mon[31423]: pgmap v71: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-04-21T10:33:15.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:15 smithi026 ceph-mon[27965]: pgmap v71: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-04-21T10:33:15.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:15 smithi169 ceph-mon[31469]: pgmap v71: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-04-21T10:33:17.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:17 smithi190 ceph-mon[31423]: pgmap v72: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-04-21T10:33:17.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:17 smithi026 ceph-mon[27965]: pgmap v72: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-04-21T10:33:17.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:17 smithi169 ceph-mon[31469]: pgmap v72: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-04-21T10:33:19.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:19 smithi190 ceph-mon[31423]: pgmap v73: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-21T10:33:19.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:19 smithi026 ceph-mon[27965]: pgmap v73: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-21T10:33:19.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:19 smithi169 ceph-mon[31469]: pgmap v73: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-21T10:33:21.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:21 smithi190 ceph-mon[31423]: pgmap v74: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-21T10:33:21.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:21 smithi026 ceph-mon[27965]: pgmap v74: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-21T10:33:21.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:21 smithi169 ceph-mon[31469]: pgmap v74: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-21T10:33:23.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:23 smithi190 ceph-mon[31423]: pgmap v75: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-21T10:33:23.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:23 smithi026 ceph-mon[27965]: pgmap v75: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-21T10:33:23.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:23 smithi169 ceph-mon[31469]: pgmap v75: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-21T10:33:25.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:25 smithi190 ceph-mon[31423]: pgmap v76: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-21T10:33:25.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:25 smithi026 ceph-mon[27965]: pgmap v76: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-21T10:33:25.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:25 smithi169 ceph-mon[31469]: pgmap v76: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-21T10:33:27.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:27 smithi190 ceph-mon[31423]: pgmap v77: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-04-21T10:33:27.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:27 smithi026 ceph-mon[27965]: pgmap v77: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-04-21T10:33:27.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:27 smithi169 ceph-mon[31469]: pgmap v77: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-04-21T10:33:29.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:29 smithi190 ceph-mon[31423]: pgmap v78: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-21T10:33:29.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:29 smithi026 ceph-mon[27965]: pgmap v78: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-21T10:33:29.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:29 smithi169 ceph-mon[31469]: pgmap v78: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-21T10:33:31.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:31 smithi190 ceph-mon[31423]: pgmap v79: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-04-21T10:33:31.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:31 smithi026 ceph-mon[27965]: pgmap v79: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-04-21T10:33:31.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:31 smithi169 ceph-mon[31469]: pgmap v79: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-04-21T10:33:33.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:33 smithi026 ceph-mon[27965]: pgmap v80: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-21T10:33:33.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:33 smithi169 ceph-mon[31469]: pgmap v80: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-21T10:33:34.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:33 smithi190 ceph-mon[31423]: pgmap v80: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-21T10:33:35.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:35 smithi026 ceph-mon[27965]: pgmap v81: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-21T10:33:35.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:35 smithi169 ceph-mon[31469]: pgmap v81: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-21T10:33:36.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:35 smithi190 ceph-mon[31423]: pgmap v81: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-21T10:33:37.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:37 smithi026 ceph-mon[27965]: pgmap v82: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-21T10:33:37.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:37 smithi169 ceph-mon[31469]: pgmap v82: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-21T10:33:38.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:37 smithi190 ceph-mon[31423]: pgmap v82: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-21T10:33:39.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:39 smithi026 ceph-mon[27965]: pgmap v83: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-21T10:33:39.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:39 smithi169 ceph-mon[31469]: pgmap v83: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-21T10:33:40.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:39 smithi190 ceph-mon[31423]: pgmap v83: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-21T10:33:40.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:40 smithi026 ceph-mon[27965]: pgmap v84: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-21T10:33:40.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:40 smithi169 ceph-mon[31469]: pgmap v84: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-21T10:33:41.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:40 smithi190 ceph-mon[31423]: pgmap v84: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-21T10:33:43.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:42 smithi026 ceph-mon[27965]: pgmap v85: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-21T10:33:43.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:42 smithi169 ceph-mon[31469]: pgmap v85: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-21T10:33:43.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:42 smithi190 ceph-mon[31423]: pgmap v85: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-21T10:33:45.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:44 smithi026 ceph-mon[27965]: pgmap v86: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-21T10:33:45.197 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:44 smithi169 ceph-mon[31469]: pgmap v86: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-21T10:33:45.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:44 smithi190 ceph-mon[31423]: pgmap v86: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-21T10:33:47.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:46 smithi026 ceph-mon[27965]: pgmap v87: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-21T10:33:47.197 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:46 smithi169 ceph-mon[31469]: pgmap v87: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-21T10:33:47.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:46 smithi190 ceph-mon[31423]: pgmap v87: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-21T10:33:49.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:48 smithi026 ceph-mon[27965]: pgmap v88: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T10:33:49.197 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:48 smithi169 ceph-mon[31469]: pgmap v88: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T10:33:49.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:48 smithi190 ceph-mon[31423]: pgmap v88: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T10:33:51.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:50 smithi026 ceph-mon[27965]: pgmap v89: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-04-21T10:33:51.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:50 smithi169 ceph-mon[31469]: pgmap v89: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-04-21T10:33:51.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:50 smithi190 ceph-mon[31423]: pgmap v89: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-04-21T10:33:53.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:52 smithi026 ceph-mon[27965]: pgmap v90: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.4 KiB/s wr, 5 op/s 2024-04-21T10:33:53.197 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:52 smithi169 ceph-mon[31469]: pgmap v90: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.4 KiB/s wr, 5 op/s 2024-04-21T10:33:53.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:52 smithi190 ceph-mon[31423]: pgmap v90: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.4 KiB/s wr, 5 op/s 2024-04-21T10:33:55.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:54 smithi026 ceph-mon[27965]: pgmap v91: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.4 KiB/s wr, 5 op/s 2024-04-21T10:33:55.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:54 smithi169 ceph-mon[31469]: pgmap v91: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.4 KiB/s wr, 5 op/s 2024-04-21T10:33:55.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:54 smithi190 ceph-mon[31423]: pgmap v91: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.4 KiB/s wr, 5 op/s 2024-04-21T10:33:57.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:56 smithi026 ceph-mon[27965]: pgmap v92: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-04-21T10:33:57.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:56 smithi169 ceph-mon[31469]: pgmap v92: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-04-21T10:33:57.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:56 smithi190 ceph-mon[31423]: pgmap v92: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-04-21T10:33:59.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:33:58 smithi026 ceph-mon[27965]: pgmap v93: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 6 op/s 2024-04-21T10:33:59.197 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:33:58 smithi169 ceph-mon[31469]: pgmap v93: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 6 op/s 2024-04-21T10:33:59.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:33:58 smithi190 ceph-mon[31423]: pgmap v93: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 6 op/s 2024-04-21T10:34:01.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:00 smithi169 ceph-mon[31469]: pgmap v94: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 8 op/s 2024-04-21T10:34:01.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:00 smithi190 ceph-mon[31423]: pgmap v94: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 8 op/s 2024-04-21T10:34:01.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:00 smithi026 ceph-mon[27965]: pgmap v94: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 8 op/s 2024-04-21T10:34:03.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:02 smithi169 ceph-mon[31469]: pgmap v95: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-04-21T10:34:03.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:02 smithi190 ceph-mon[31423]: pgmap v95: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-04-21T10:34:03.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:02 smithi026 ceph-mon[27965]: pgmap v95: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-04-21T10:34:05.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:04 smithi169 ceph-mon[31469]: pgmap v96: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-04-21T10:34:05.237 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:04 smithi026 ceph-mon[27965]: pgmap v96: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-04-21T10:34:05.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:04 smithi190 ceph-mon[31423]: pgmap v96: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-04-21T10:34:06.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:05 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:34:06.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:05 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:34:06.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:05 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:34:07.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:06 smithi190 ceph-mon[31423]: pgmap v97: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-21T10:34:07.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:06 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:34:07.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:06 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:34:07.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:06 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:34:07.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:06 smithi026 ceph-mon[27965]: pgmap v97: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-21T10:34:07.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:06 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:34:07.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:06 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:34:07.436 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:06 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:34:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:06 smithi169 ceph-mon[31469]: pgmap v97: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-21T10:34:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:06 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:34:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:06 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:34:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:06 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:34:09.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:08 smithi190 ceph-mon[31423]: pgmap v98: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-04-21T10:34:09.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:08 smithi026 ceph-mon[27965]: pgmap v98: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-04-21T10:34:09.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:08 smithi169 ceph-mon[31469]: pgmap v98: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-04-21T10:34:10.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:09 smithi190 ceph-mon[31423]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:34:10.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:09 smithi026 ceph-mon[27965]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:34:10.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:09 smithi169 ceph-mon[31469]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:34:11.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:10 smithi190 ceph-mon[31423]: pgmap v99: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-21T10:34:11.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:10 smithi026 ceph-mon[27965]: pgmap v99: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-21T10:34:11.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:10 smithi169 ceph-mon[31469]: pgmap v99: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-21T10:34:13.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:12 smithi190 ceph-mon[31423]: pgmap v100: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-04-21T10:34:13.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:12 smithi026 ceph-mon[27965]: pgmap v100: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-04-21T10:34:13.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:12 smithi169 ceph-mon[31469]: pgmap v100: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-04-21T10:34:15.249 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:14 smithi169 ceph-mon[31469]: pgmap v101: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-04-21T10:34:15.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:14 smithi190 ceph-mon[31423]: pgmap v101: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-04-21T10:34:15.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:14 smithi026 ceph-mon[27965]: pgmap v101: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-04-21T10:34:17.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:16 smithi190 ceph-mon[31423]: pgmap v102: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-21T10:34:17.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:16 smithi026 ceph-mon[27965]: pgmap v102: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-21T10:34:17.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:16 smithi169 ceph-mon[31469]: pgmap v102: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-21T10:34:19.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:18 smithi190 ceph-mon[31423]: pgmap v103: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-21T10:34:19.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:18 smithi026 ceph-mon[27965]: pgmap v103: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-21T10:34:19.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:18 smithi169 ceph-mon[31469]: pgmap v103: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-21T10:34:21.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:20 smithi190 ceph-mon[31423]: pgmap v104: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-04-21T10:34:21.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:20 smithi026 ceph-mon[27965]: pgmap v104: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-04-21T10:34:21.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:20 smithi169 ceph-mon[31469]: pgmap v104: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-04-21T10:34:23.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:22 smithi190 ceph-mon[31423]: pgmap v105: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-21T10:34:23.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:22 smithi026 ceph-mon[27965]: pgmap v105: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-21T10:34:23.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:22 smithi169 ceph-mon[31469]: pgmap v105: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-21T10:34:25.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:24 smithi190 ceph-mon[31423]: pgmap v106: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-21T10:34:25.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:24 smithi026 ceph-mon[27965]: pgmap v106: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-21T10:34:25.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:24 smithi169 ceph-mon[31469]: pgmap v106: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-21T10:34:27.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:26 smithi190 ceph-mon[31423]: pgmap v107: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-04-21T10:34:27.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:26 smithi026 ceph-mon[27965]: pgmap v107: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-04-21T10:34:27.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:26 smithi169 ceph-mon[31469]: pgmap v107: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-04-21T10:34:29.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:28 smithi190 ceph-mon[31423]: pgmap v108: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-04-21T10:34:29.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:28 smithi026 ceph-mon[27965]: pgmap v108: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-04-21T10:34:29.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:28 smithi169 ceph-mon[31469]: pgmap v108: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-04-21T10:34:31.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:30 smithi190 ceph-mon[31423]: pgmap v109: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-21T10:34:31.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:30 smithi026 ceph-mon[27965]: pgmap v109: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-21T10:34:31.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:30 smithi169 ceph-mon[31469]: pgmap v109: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-21T10:34:33.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:33 smithi190 ceph-mon[31423]: pgmap v110: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-21T10:34:33.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:33 smithi026 ceph-mon[27965]: pgmap v110: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-21T10:34:33.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:33 smithi169 ceph-mon[31469]: pgmap v110: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-21T10:34:35.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:35 smithi190 ceph-mon[31423]: pgmap v111: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-21T10:34:35.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:35 smithi026 ceph-mon[27965]: pgmap v111: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-21T10:34:35.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:35 smithi169 ceph-mon[31469]: pgmap v111: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-21T10:34:37.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:37 smithi190 ceph-mon[31423]: pgmap v112: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-21T10:34:37.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:37 smithi026 ceph-mon[27965]: pgmap v112: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-21T10:34:37.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:37 smithi169 ceph-mon[31469]: pgmap v112: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-21T10:34:39.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:39 smithi190 ceph-mon[31423]: pgmap v113: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-21T10:34:39.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:39 smithi026 ceph-mon[27965]: pgmap v113: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-21T10:34:39.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:39 smithi169 ceph-mon[31469]: pgmap v113: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-21T10:34:41.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:41 smithi190 ceph-mon[31423]: pgmap v114: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-21T10:34:41.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:41 smithi026 ceph-mon[27965]: pgmap v114: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-21T10:34:41.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:41 smithi169 ceph-mon[31469]: pgmap v114: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-21T10:34:43.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:43 smithi190 ceph-mon[31423]: pgmap v115: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:43.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:43 smithi026 ceph-mon[27965]: pgmap v115: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:43.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:43 smithi169 ceph-mon[31469]: pgmap v115: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:45.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:45 smithi190 ceph-mon[31423]: pgmap v116: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:45.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:45 smithi026 ceph-mon[27965]: pgmap v116: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:45.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:45 smithi169 ceph-mon[31469]: pgmap v116: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:47.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:47 smithi190 ceph-mon[31423]: pgmap v117: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:34:47.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:47 smithi026 ceph-mon[27965]: pgmap v117: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:34:47.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:47 smithi169 ceph-mon[31469]: pgmap v117: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:34:49.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:49 smithi026 ceph-mon[27965]: pgmap v118: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:49.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:49 smithi026 ceph-mon[27965]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:34:49.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:49 smithi169 ceph-mon[31469]: pgmap v118: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:49.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:49 smithi169 ceph-mon[31469]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:34:49.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:49 smithi190 ceph-mon[31423]: pgmap v118: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:49 smithi190 ceph-mon[31423]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:34:51.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:51 smithi026 ceph-mon[27965]: pgmap v119: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:34:51.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:51 smithi169 ceph-mon[31469]: pgmap v119: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:34:51.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:51 smithi190 ceph-mon[31423]: pgmap v119: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:34:53.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:53 smithi026 ceph-mon[27965]: pgmap v120: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:53.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:53 smithi169 ceph-mon[31469]: pgmap v120: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:53.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:53 smithi190 ceph-mon[31423]: pgmap v120: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:55.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:55 smithi026 ceph-mon[27965]: pgmap v121: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:55.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:55 smithi169 ceph-mon[31469]: pgmap v121: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:55.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:55 smithi190 ceph-mon[31423]: pgmap v121: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:57.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:57 smithi026 ceph-mon[27965]: pgmap v122: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:34:57.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:57 smithi169 ceph-mon[31469]: pgmap v122: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:34:57.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:57 smithi190 ceph-mon[31423]: pgmap v122: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:34:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:34:59 smithi026 ceph-mon[27965]: pgmap v123: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:59.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:34:59 smithi169 ceph-mon[31469]: pgmap v123: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:34:59.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:34:59 smithi190 ceph-mon[31423]: pgmap v123: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:01.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:01 smithi026 ceph-mon[27965]: pgmap v124: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:01.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:01 smithi169 ceph-mon[31469]: pgmap v124: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:01.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:01 smithi190 ceph-mon[31423]: pgmap v124: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:03.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:03 smithi026 ceph-mon[27965]: pgmap v125: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:03.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:03 smithi169 ceph-mon[31469]: pgmap v125: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:03.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:03 smithi190 ceph-mon[31423]: pgmap v125: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:05.438 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:05 smithi026 ceph-mon[27965]: pgmap v126: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:05.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:05 smithi169 ceph-mon[31469]: pgmap v126: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:05.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:05 smithi190 ceph-mon[31423]: pgmap v126: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:07.054 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:07 smithi026 ceph-mon[27965]: pgmap v127: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:35:07.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:07 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:35:07.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:07 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:35:07.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:07 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:35:07.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:07 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:35:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:07 smithi169 ceph-mon[31469]: pgmap v127: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:35:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:07 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:35:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:07 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:35:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:07 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:35:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:07 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:35:07.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:07 smithi190 ceph-mon[31423]: pgmap v127: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:35:07.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:07 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:35:07.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:07 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:35:07.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:07 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:35:07.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:07 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:35:09.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:09 smithi026 ceph-mon[27965]: pgmap v128: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:35:09.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:09 smithi169 ceph-mon[31469]: pgmap v128: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:35:09.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:09 smithi190 ceph-mon[31423]: pgmap v128: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:35:11.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:11 smithi026 ceph-mon[27965]: pgmap v129: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:35:11.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:11 smithi169 ceph-mon[31469]: pgmap v129: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:35:11.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:11 smithi190 ceph-mon[31423]: pgmap v129: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:35:13.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:13 smithi026 ceph-mon[27965]: pgmap v130: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:35:13.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:13 smithi169 ceph-mon[31469]: pgmap v130: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:35:13.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:13 smithi190 ceph-mon[31423]: pgmap v130: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:35:15.329 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:15 smithi190 ceph-mon[31423]: pgmap v131: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:35:15.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:15 smithi026 ceph-mon[27965]: pgmap v131: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:35:15.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:15 smithi169 ceph-mon[31469]: pgmap v131: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:35:17.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:17 smithi026 ceph-mon[27965]: pgmap v132: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:35:17.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:17 smithi169 ceph-mon[31469]: pgmap v132: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:35:17.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:17 smithi190 ceph-mon[31423]: pgmap v132: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:35:19.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:19 smithi026 ceph-mon[27965]: pgmap v133: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T10:35:19.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:19 smithi169 ceph-mon[31469]: pgmap v133: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T10:35:19.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:19 smithi190 ceph-mon[31423]: pgmap v133: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T10:35:21.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:21 smithi026 ceph-mon[27965]: pgmap v134: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T10:35:21.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:21 smithi169 ceph-mon[31469]: pgmap v134: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T10:35:21.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:21 smithi190 ceph-mon[31423]: pgmap v134: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T10:35:23.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:23 smithi026 ceph-mon[27965]: pgmap v135: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:23.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:23 smithi169 ceph-mon[31469]: pgmap v135: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:23.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:23 smithi190 ceph-mon[31423]: pgmap v135: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:25.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:25 smithi026 ceph-mon[27965]: pgmap v136: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:25.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:25 smithi169 ceph-mon[31469]: pgmap v136: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:25.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:25 smithi190 ceph-mon[31423]: pgmap v136: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:27.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:27 smithi026 ceph-mon[27965]: pgmap v137: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:27.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:27 smithi169 ceph-mon[31469]: pgmap v137: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:27.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:27 smithi190 ceph-mon[31423]: pgmap v137: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:29.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:29 smithi026 ceph-mon[27965]: pgmap v138: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:35:29.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:29 smithi026 ceph-mon[27965]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:35:29.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:29 smithi169 ceph-mon[31469]: pgmap v138: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:35:29.474 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:29 smithi169 ceph-mon[31469]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:35:29.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:29 smithi190 ceph-mon[31423]: pgmap v138: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:35:29.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:29 smithi190 ceph-mon[31423]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T10:35:31.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:31 smithi026 ceph-mon[27965]: pgmap v139: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:31.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:31 smithi169 ceph-mon[31469]: pgmap v139: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:31.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:31 smithi190 ceph-mon[31423]: pgmap v139: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:33.346 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:33 smithi190 ceph-mon[31423]: pgmap v140: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:35:33.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:33 smithi026 ceph-mon[27965]: pgmap v140: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:35:33.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:33 smithi169 ceph-mon[31469]: pgmap v140: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:35:35.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:35 smithi026 ceph-mon[27965]: pgmap v141: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:35:35.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:35 smithi169 ceph-mon[31469]: pgmap v141: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:35:35.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:35 smithi190 ceph-mon[31423]: pgmap v141: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:35:37.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:37 smithi026 ceph-mon[27965]: pgmap v142: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:37.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:37 smithi169 ceph-mon[31469]: pgmap v142: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:37.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:37 smithi190 ceph-mon[31423]: pgmap v142: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:39.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:39 smithi026 ceph-mon[27965]: pgmap v143: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:39 smithi169 ceph-mon[31469]: pgmap v143: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:39.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:39 smithi190 ceph-mon[31423]: pgmap v143: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:41.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:41 smithi026 ceph-mon[27965]: pgmap v144: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:41.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:41 smithi169 ceph-mon[31469]: pgmap v144: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:41.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:41 smithi190 ceph-mon[31423]: pgmap v144: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:43.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:43 smithi026 ceph-mon[27965]: pgmap v145: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:43.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:43 smithi169 ceph-mon[31469]: pgmap v145: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:43.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:43 smithi190 ceph-mon[31423]: pgmap v145: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:45.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:45 smithi026 ceph-mon[27965]: pgmap v146: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:45.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:45 smithi169 ceph-mon[31469]: pgmap v146: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:45.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:45 smithi190 ceph-mon[31423]: pgmap v146: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:47.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:47 smithi026 ceph-mon[27965]: pgmap v147: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:47.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:47 smithi169 ceph-mon[31469]: pgmap v147: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:47.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:47 smithi190 ceph-mon[31423]: pgmap v147: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:49.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:49 smithi026 ceph-mon[27965]: pgmap v148: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:49.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:49 smithi169 ceph-mon[31469]: pgmap v148: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:49 smithi190 ceph-mon[31423]: pgmap v148: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:51.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:51 smithi026 ceph-mon[27965]: pgmap v149: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:51.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:51 smithi169 ceph-mon[31469]: pgmap v149: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:51.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:51 smithi190 ceph-mon[31423]: pgmap v149: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:53.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:53 smithi026 ceph-mon[27965]: pgmap v150: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:53.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:53 smithi169 ceph-mon[31469]: pgmap v150: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:53.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:53 smithi190 ceph-mon[31423]: pgmap v150: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:55.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:55 smithi026 ceph-mon[27965]: pgmap v151: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:55.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:55 smithi169 ceph-mon[31469]: pgmap v151: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:55.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:55 smithi190 ceph-mon[31423]: pgmap v151: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:57.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:57 smithi026 ceph-mon[27965]: pgmap v152: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:57.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:57 smithi169 ceph-mon[31469]: pgmap v152: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:57.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:57 smithi190 ceph-mon[31423]: pgmap v152: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:35:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:35:59 smithi026 ceph-mon[27965]: pgmap v153: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:59.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:35:59 smithi169 ceph-mon[31469]: pgmap v153: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:35:59.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:35:59 smithi190 ceph-mon[31423]: pgmap v153: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:01.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:01 smithi026 ceph-mon[27965]: pgmap v154: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:01.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:01 smithi169 ceph-mon[31469]: pgmap v154: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:01.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:01 smithi190 ceph-mon[31423]: pgmap v154: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:03.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:03 smithi026 ceph-mon[27965]: pgmap v155: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:03.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:03 smithi169 ceph-mon[31469]: pgmap v155: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:03.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:03 smithi190 ceph-mon[31423]: pgmap v155: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:05.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:05 smithi026 ceph-mon[27965]: pgmap v156: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-21T10:36:05.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:05 smithi169 ceph-mon[31469]: pgmap v156: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-21T10:36:05.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:05 smithi190 ceph-mon[31423]: pgmap v156: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-21T10:36:07.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:07 smithi026 ceph-mon[27965]: pgmap v157: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:36:07.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:07 smithi169 ceph-mon[31469]: pgmap v157: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:36:07.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:07 smithi190 ceph-mon[31423]: pgmap v157: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:36:08.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:08 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:36:08.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:08 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:36:08.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:08 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:36:08.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:08 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:36:08.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:08 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:36:08.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:08 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:36:08.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:08 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:36:08.449 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:08 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:36:08.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:08 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:36:08.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:08 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:36:08.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:08 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:36:08.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:08 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:36:09.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:09 smithi026 ceph-mon[27965]: pgmap v158: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:36:09.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:09 smithi169 ceph-mon[31469]: pgmap v158: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:36:09.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:09 smithi190 ceph-mon[31423]: pgmap v158: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:36:11.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:11 smithi026 ceph-mon[27965]: pgmap v159: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:36:11.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:11 smithi169 ceph-mon[31469]: pgmap v159: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:36:11.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:11 smithi190 ceph-mon[31423]: pgmap v159: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-21T10:36:13.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:13 smithi026 ceph-mon[27965]: pgmap v160: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:36:13.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:13 smithi169 ceph-mon[31469]: pgmap v160: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:36:13.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:13 smithi190 ceph-mon[31423]: pgmap v160: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:36:15.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:15 smithi026 ceph-mon[27965]: pgmap v161: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:36:15.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:15 smithi169 ceph-mon[31469]: pgmap v161: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:36:15.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:15 smithi190 ceph-mon[31423]: pgmap v161: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-21T10:36:17.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:17 smithi026 ceph-mon[27965]: pgmap v162: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-04-21T10:36:17.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:17 smithi169 ceph-mon[31469]: pgmap v162: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-04-21T10:36:17.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:17 smithi190 ceph-mon[31423]: pgmap v162: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-04-21T10:36:19.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:19 smithi026 ceph-mon[27965]: pgmap v163: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T10:36:19.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:19 smithi169 ceph-mon[31469]: pgmap v163: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T10:36:19.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:19 smithi190 ceph-mon[31423]: pgmap v163: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-21T10:36:21.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:21 smithi026 ceph-mon[27965]: pgmap v164: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T10:36:21.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:21 smithi169 ceph-mon[31469]: pgmap v164: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T10:36:21.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:21 smithi190 ceph-mon[31423]: pgmap v164: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T10:36:23.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:23 smithi026 ceph-mon[27965]: pgmap v165: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:23.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:23 smithi169 ceph-mon[31469]: pgmap v165: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:23.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:23 smithi190 ceph-mon[31423]: pgmap v165: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:25.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:25 smithi026 ceph-mon[27965]: pgmap v166: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:25.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:25 smithi169 ceph-mon[31469]: pgmap v166: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:25.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:25 smithi190 ceph-mon[31423]: pgmap v166: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:27.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:27 smithi026 ceph-mon[27965]: pgmap v167: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:27.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:27 smithi169 ceph-mon[31469]: pgmap v167: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:27.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:27 smithi190 ceph-mon[31423]: pgmap v167: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:29.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:29 smithi026 ceph-mon[27965]: pgmap v168: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:29.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:29 smithi169 ceph-mon[31469]: pgmap v168: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:29.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:29 smithi190 ceph-mon[31423]: pgmap v168: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:31.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:31 smithi026 ceph-mon[27965]: pgmap v169: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:31.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:31 smithi169 ceph-mon[31469]: pgmap v169: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:31.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:31 smithi190 ceph-mon[31423]: pgmap v169: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:33.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:33 smithi026 ceph-mon[27965]: pgmap v170: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:33.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:33 smithi169 ceph-mon[31469]: pgmap v170: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:33.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:33 smithi190 ceph-mon[31423]: pgmap v170: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:35.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:35 smithi169 ceph-mon[31469]: pgmap v171: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:35.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:35 smithi190 ceph-mon[31423]: pgmap v171: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:35.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:35 smithi026 ceph-mon[27965]: pgmap v171: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:37.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:37 smithi169 ceph-mon[31469]: pgmap v172: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:37.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:37 smithi190 ceph-mon[31423]: pgmap v172: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:37.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:37 smithi026 ceph-mon[27965]: pgmap v172: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:39.450 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:39 smithi169 ceph-mon[31469]: pgmap v173: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:39.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:39 smithi190 ceph-mon[31423]: pgmap v173: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:39.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:39 smithi026 ceph-mon[27965]: pgmap v173: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:41.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:41 smithi169 ceph-mon[31469]: pgmap v174: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:41.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:41 smithi190 ceph-mon[31423]: pgmap v174: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:41 smithi026 ceph-mon[27965]: pgmap v174: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:43.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:43 smithi169 ceph-mon[31469]: pgmap v175: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:43.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:43 smithi190 ceph-mon[31423]: pgmap v175: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:43.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:43 smithi026 ceph-mon[27965]: pgmap v175: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:45.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:45 smithi190 ceph-mon[31423]: pgmap v176: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:45.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:45 smithi026 ceph-mon[27965]: pgmap v176: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:45.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:45 smithi169 ceph-mon[31469]: pgmap v176: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:47.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:47 smithi190 ceph-mon[31423]: pgmap v177: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:47.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:47 smithi026 ceph-mon[27965]: pgmap v177: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:47.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:47 smithi169 ceph-mon[31469]: pgmap v177: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:49.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:49 smithi190 ceph-mon[31423]: pgmap v178: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:49 smithi026 ceph-mon[27965]: pgmap v178: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:49.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:49 smithi169 ceph-mon[31469]: pgmap v178: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:51.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:51 smithi190 ceph-mon[31423]: pgmap v179: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:51.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:51 smithi026 ceph-mon[27965]: pgmap v179: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:51.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:51 smithi169 ceph-mon[31469]: pgmap v179: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:53.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:53 smithi190 ceph-mon[31423]: pgmap v180: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:53.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:53 smithi026 ceph-mon[27965]: pgmap v180: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:53.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:53 smithi169 ceph-mon[31469]: pgmap v180: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:55.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:55 smithi190 ceph-mon[31423]: pgmap v181: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:55.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:55 smithi026 ceph-mon[27965]: pgmap v181: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:55.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:55 smithi169 ceph-mon[31469]: pgmap v181: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:57.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:57 smithi190 ceph-mon[31423]: pgmap v182: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:57.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:57 smithi026 ceph-mon[27965]: pgmap v182: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:57.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:57 smithi169 ceph-mon[31469]: pgmap v182: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:36:59.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:36:59 smithi190 ceph-mon[31423]: pgmap v183: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:59.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:36:59 smithi026 ceph-mon[27965]: pgmap v183: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:36:59.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:36:59 smithi169 ceph-mon[31469]: pgmap v183: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:01.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:01 smithi190 ceph-mon[31423]: pgmap v184: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:01.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:01 smithi026 ceph-mon[27965]: pgmap v184: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:01.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:01 smithi169 ceph-mon[31469]: pgmap v184: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:03.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:03 smithi190 ceph-mon[31423]: pgmap v185: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:03.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:03 smithi026 ceph-mon[27965]: pgmap v185: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:03.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:03 smithi169 ceph-mon[31469]: pgmap v185: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:05.487 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:05 smithi026 ceph-mon[27965]: pgmap v186: 97 pgs: 97 active+clean; 65 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-04-21T10:37:05.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:05 smithi190 ceph-mon[31423]: pgmap v186: 97 pgs: 97 active+clean; 65 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-04-21T10:37:05.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:05 smithi169 ceph-mon[31469]: pgmap v186: 97 pgs: 97 active+clean; 65 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-04-21T10:37:07.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:07 smithi190 ceph-mon[31423]: pgmap v187: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-21T10:37:07.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:07 smithi026 ceph-mon[27965]: pgmap v187: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-21T10:37:07.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:07 smithi169 ceph-mon[31469]: pgmap v187: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-21T10:37:08.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:08 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:37:08.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:08 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:37:08.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:08 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:37:09.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:09 smithi190 ceph-mon[31423]: pgmap v188: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-21T10:37:09.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:09 smithi190 ceph-mon[31423]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T10:37:09.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:09 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:37:09.530 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:09 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:37:09.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:09 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:37:09.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:09 smithi026 ceph-mon[27965]: pgmap v188: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-21T10:37:09.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:09 smithi026 ceph-mon[27965]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T10:37:09.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:09 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:37:09.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:09 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:37:09.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:09 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:37:09.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:09 smithi169 ceph-mon[31469]: pgmap v188: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-21T10:37:09.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:09 smithi169 ceph-mon[31469]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T10:37:09.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:09 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:37:09.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:09 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:37:09.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:09 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:37:11.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:11 smithi190 ceph-mon[31423]: pgmap v190: 97 pgs: 97 active+clean; 65 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-21T10:37:11.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:11 smithi026 ceph-mon[27965]: pgmap v190: 97 pgs: 97 active+clean; 65 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-21T10:37:11.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:11 smithi169 ceph-mon[31469]: pgmap v190: 97 pgs: 97 active+clean; 65 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-21T10:37:13.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:13 smithi190 ceph-mon[31423]: pgmap v191: 97 pgs: 97 active+clean; 65 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-21T10:37:13.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:13 smithi026 ceph-mon[27965]: pgmap v191: 97 pgs: 97 active+clean; 65 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-21T10:37:13.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:13 smithi169 ceph-mon[31469]: pgmap v191: 97 pgs: 97 active+clean; 65 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-21T10:37:15.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:14 smithi026 ceph-mon[27965]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T10:37:15.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:14 smithi026 ceph-mon[27965]: pgmap v193: 97 pgs: 97 active+clean; 65 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T10:37:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:14 smithi169 ceph-mon[31469]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T10:37:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:14 smithi169 ceph-mon[31469]: pgmap v193: 97 pgs: 97 active+clean; 65 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T10:37:15.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:14 smithi190 ceph-mon[31423]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T10:37:15.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:14 smithi190 ceph-mon[31423]: pgmap v193: 97 pgs: 97 active+clean; 65 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-21T10:37:17.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:16 smithi190 ceph-mon[31423]: pgmap v194: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:37:17.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:16 smithi026 ceph-mon[27965]: pgmap v194: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:37:17.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:16 smithi169 ceph-mon[31469]: pgmap v194: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:37:19.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:18 smithi190 ceph-mon[31423]: pgmap v195: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-04-21T10:37:19.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:18 smithi026 ceph-mon[27965]: pgmap v195: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-04-21T10:37:19.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:18 smithi169 ceph-mon[31469]: pgmap v195: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-04-21T10:37:21.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:20 smithi190 ceph-mon[31423]: pgmap v196: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:37:21.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:20 smithi026 ceph-mon[27965]: pgmap v196: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:37:21.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:20 smithi169 ceph-mon[31469]: pgmap v196: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:37:23.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:22 smithi190 ceph-mon[31423]: pgmap v197: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:37:23.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:22 smithi026 ceph-mon[27965]: pgmap v197: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:37:23.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:22 smithi169 ceph-mon[31469]: pgmap v197: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:37:25.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:24 smithi190 ceph-mon[31423]: pgmap v198: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:37:25.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:24 smithi026 ceph-mon[27965]: pgmap v198: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:37:25.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:24 smithi169 ceph-mon[31469]: pgmap v198: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:37:27.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:26 smithi190 ceph-mon[31423]: pgmap v199: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T10:37:27.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:26 smithi026 ceph-mon[27965]: pgmap v199: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T10:37:27.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:26 smithi169 ceph-mon[31469]: pgmap v199: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-21T10:37:29.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:28 smithi190 ceph-mon[31423]: pgmap v200: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:29.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:28 smithi026 ceph-mon[27965]: pgmap v200: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:29.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:28 smithi169 ceph-mon[31469]: pgmap v200: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:31.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:30 smithi190 ceph-mon[31423]: pgmap v201: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:31.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:30 smithi026 ceph-mon[27965]: pgmap v201: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:31.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:30 smithi169 ceph-mon[31469]: pgmap v201: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:33.278 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:32 smithi190 ceph-mon[31423]: pgmap v202: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:33.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:32 smithi026 ceph-mon[27965]: pgmap v202: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:33.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:32 smithi169 ceph-mon[31469]: pgmap v202: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:35.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:34 smithi190 ceph-mon[31423]: pgmap v203: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:35.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:34 smithi026 ceph-mon[27965]: pgmap v203: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:35.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:34 smithi169 ceph-mon[31469]: pgmap v203: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:37.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:37 smithi190 ceph-mon[31423]: pgmap v204: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:37.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:37 smithi026 ceph-mon[27965]: pgmap v204: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:37.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:37 smithi169 ceph-mon[31469]: pgmap v204: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:39.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:39 smithi190 ceph-mon[31423]: pgmap v205: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:39.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:39 smithi026 ceph-mon[27965]: pgmap v205: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:39 smithi169 ceph-mon[31469]: pgmap v205: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:41.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:41 smithi190 ceph-mon[31423]: pgmap v206: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:41.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:41 smithi026 ceph-mon[27965]: pgmap v206: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:41.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:41 smithi169 ceph-mon[31469]: pgmap v206: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:43.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:43 smithi190 ceph-mon[31423]: pgmap v207: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:43.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:43 smithi026 ceph-mon[27965]: pgmap v207: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:43.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:43 smithi169 ceph-mon[31469]: pgmap v207: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:45.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:45 smithi190 ceph-mon[31423]: pgmap v208: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:45.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:45 smithi026 ceph-mon[27965]: pgmap v208: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:45.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:45 smithi169 ceph-mon[31469]: pgmap v208: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:47.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:47 smithi190 ceph-mon[31423]: pgmap v209: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:47.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:47 smithi026 ceph-mon[27965]: pgmap v209: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:47.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:47 smithi169 ceph-mon[31469]: pgmap v209: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:49.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:49 smithi190 ceph-mon[31423]: pgmap v210: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:49.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:49 smithi026 ceph-mon[27965]: pgmap v210: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:49.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:49 smithi169 ceph-mon[31469]: pgmap v210: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:51.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:51 smithi190 ceph-mon[31423]: pgmap v211: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:51.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:51 smithi026 ceph-mon[27965]: pgmap v211: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:51.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:51 smithi169 ceph-mon[31469]: pgmap v211: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:53.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:53 smithi190 ceph-mon[31423]: pgmap v212: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:53.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:53 smithi026 ceph-mon[27965]: pgmap v212: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:53.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:53 smithi169 ceph-mon[31469]: pgmap v212: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:55.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:55 smithi190 ceph-mon[31423]: pgmap v213: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:55.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:55 smithi026 ceph-mon[27965]: pgmap v213: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:55.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:55 smithi169 ceph-mon[31469]: pgmap v213: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:57.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:57 smithi026 ceph-mon[27965]: pgmap v214: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:57.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:57 smithi169 ceph-mon[31469]: pgmap v214: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:57.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:57 smithi190 ceph-mon[31423]: pgmap v214: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:37:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:37:59 smithi026 ceph-mon[27965]: pgmap v215: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:59.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:37:59 smithi169 ceph-mon[31469]: pgmap v215: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:37:59.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:37:59 smithi190 ceph-mon[31423]: pgmap v215: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:01.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:01 smithi026 ceph-mon[27965]: pgmap v216: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:38:01.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:01 smithi169 ceph-mon[31469]: pgmap v216: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:38:01.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:01 smithi190 ceph-mon[31423]: pgmap v216: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:38:03.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:03 smithi026 ceph-mon[27965]: pgmap v217: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:03.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:03 smithi169 ceph-mon[31469]: pgmap v217: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:03.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:03 smithi190 ceph-mon[31423]: pgmap v217: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:05.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:05 smithi026 ceph-mon[27965]: pgmap v218: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-04-21T10:38:05.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:05 smithi169 ceph-mon[31469]: pgmap v218: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-04-21T10:38:05.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:05 smithi190 ceph-mon[31423]: pgmap v218: 97 pgs: 97 active+clean; 65 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-04-21T10:38:07.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:07 smithi026 ceph-mon[27965]: pgmap v219: 97 pgs: 97 active+clean; 65 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T10:38:07.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:07 smithi169 ceph-mon[31469]: pgmap v219: 97 pgs: 97 active+clean; 65 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T10:38:07.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:07 smithi190 ceph-mon[31423]: pgmap v219: 97 pgs: 97 active+clean; 65 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T10:38:09.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:09 smithi026 ceph-mon[27965]: pgmap v220: 97 pgs: 97 active+clean; 65 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T10:38:09.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:09 smithi026 ceph-mon[27965]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T10:38:09.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:09 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:38:09.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:09 smithi169 ceph-mon[31469]: pgmap v220: 97 pgs: 97 active+clean; 65 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T10:38:09.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:09 smithi169 ceph-mon[31469]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T10:38:09.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:09 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:38:09.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:09 smithi190 ceph-mon[31423]: pgmap v220: 97 pgs: 97 active+clean; 65 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T10:38:09.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:09 smithi190 ceph-mon[31423]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T10:38:09.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:09 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:38:10.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:10 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:38:10.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:10 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:38:10.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:10 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:38:10.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:10 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:38:10.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:10 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:38:10.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:10 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:38:10.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:10 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:38:10.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:10 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:38:10.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:10 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:38:11.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:11 smithi026 ceph-mon[27965]: pgmap v222: 97 pgs: 97 active+clean; 65 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T10:38:11.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:11 smithi169 ceph-mon[31469]: pgmap v222: 97 pgs: 97 active+clean; 65 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T10:38:11.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:11 smithi190 ceph-mon[31423]: pgmap v222: 97 pgs: 97 active+clean; 65 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T10:38:13.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:13 smithi026 ceph-mon[27965]: pgmap v223: 97 pgs: 97 active+clean; 65 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T10:38:13.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:13 smithi169 ceph-mon[31469]: pgmap v223: 97 pgs: 97 active+clean; 65 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T10:38:13.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:13 smithi190 ceph-mon[31423]: pgmap v223: 97 pgs: 97 active+clean; 65 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T10:38:15.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:14 smithi026 ceph-mon[27965]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T10:38:15.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:14 smithi026 ceph-mon[27965]: pgmap v225: 97 pgs: 97 active+clean; 65 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-21T10:38:15.197 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:14 smithi169 ceph-mon[31469]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T10:38:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:14 smithi169 ceph-mon[31469]: pgmap v225: 97 pgs: 97 active+clean; 65 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-21T10:38:15.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:14 smithi190 ceph-mon[31423]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T10:38:15.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:14 smithi190 ceph-mon[31423]: pgmap v225: 97 pgs: 97 active+clean; 65 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-21T10:38:17.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:16 smithi190 ceph-mon[31423]: pgmap v226: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:38:17.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:16 smithi026 ceph-mon[27965]: pgmap v226: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:38:17.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:16 smithi169 ceph-mon[31469]: pgmap v226: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:38:19.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:18 smithi190 ceph-mon[31423]: pgmap v227: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-21T10:38:19.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:18 smithi026 ceph-mon[27965]: pgmap v227: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-21T10:38:19.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:18 smithi169 ceph-mon[31469]: pgmap v227: 97 pgs: 97 active+clean; 65 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-21T10:38:21.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:20 smithi190 ceph-mon[31423]: pgmap v228: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:38:21.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:20 smithi026 ceph-mon[27965]: pgmap v228: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:38:21.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:20 smithi169 ceph-mon[31469]: pgmap v228: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:38:23.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:22 smithi190 ceph-mon[31423]: pgmap v229: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:38:23.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:22 smithi026 ceph-mon[27965]: pgmap v229: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:38:23.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:22 smithi169 ceph-mon[31469]: pgmap v229: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:38:25.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:25 smithi190 ceph-mon[31423]: pgmap v230: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-21T10:38:25.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:25 smithi026 ceph-mon[27965]: pgmap v230: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-21T10:38:25.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:25 smithi169 ceph-mon[31469]: pgmap v230: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-21T10:38:27.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:27 smithi190 ceph-mon[31423]: pgmap v231: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T10:38:27.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:27 smithi026 ceph-mon[27965]: pgmap v231: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T10:38:27.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:27 smithi169 ceph-mon[31469]: pgmap v231: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T10:38:29.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:29 smithi190 ceph-mon[31423]: pgmap v232: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:29.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:29 smithi026 ceph-mon[27965]: pgmap v232: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:29.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:29 smithi169 ceph-mon[31469]: pgmap v232: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:31.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:31 smithi190 ceph-mon[31423]: pgmap v233: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:38:31.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:31 smithi026 ceph-mon[27965]: pgmap v233: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:38:31.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:31 smithi169 ceph-mon[31469]: pgmap v233: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:38:33.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:33 smithi190 ceph-mon[31423]: pgmap v234: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:33.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:33 smithi026 ceph-mon[27965]: pgmap v234: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:33.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:33 smithi169 ceph-mon[31469]: pgmap v234: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:35.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:35 smithi190 ceph-mon[31423]: pgmap v235: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-21T10:38:35.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:35 smithi026 ceph-mon[27965]: pgmap v235: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-21T10:38:35.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:35 smithi169 ceph-mon[31469]: pgmap v235: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-04-21T10:38:37.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:37 smithi190 ceph-mon[31423]: pgmap v236: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T10:38:37.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:37 smithi026 ceph-mon[27965]: pgmap v236: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T10:38:37.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:37 smithi169 ceph-mon[31469]: pgmap v236: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T10:38:39.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:39 smithi190 ceph-mon[31423]: pgmap v237: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:39.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:39 smithi026 ceph-mon[27965]: pgmap v237: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:39 smithi169 ceph-mon[31469]: pgmap v237: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:41.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:41 smithi190 ceph-mon[31423]: pgmap v238: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:38:41.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:41 smithi026 ceph-mon[27965]: pgmap v238: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:38:41.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:41 smithi169 ceph-mon[31469]: pgmap v238: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:38:43.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:43 smithi190 ceph-mon[31423]: pgmap v239: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:43.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:43 smithi026 ceph-mon[27965]: pgmap v239: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:43.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:43 smithi169 ceph-mon[31469]: pgmap v239: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:45.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:45 smithi026 ceph-mon[27965]: pgmap v240: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:38:45.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:45 smithi169 ceph-mon[31469]: pgmap v240: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:38:45.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:45 smithi190 ceph-mon[31423]: pgmap v240: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:38:47.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:47 smithi026 ceph-mon[27965]: pgmap v241: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T10:38:47.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:47 smithi169 ceph-mon[31469]: pgmap v241: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T10:38:47.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:47 smithi190 ceph-mon[31423]: pgmap v241: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-21T10:38:49.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:49 smithi026 ceph-mon[27965]: pgmap v242: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:49.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:49 smithi169 ceph-mon[31469]: pgmap v242: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:49.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:49 smithi190 ceph-mon[31423]: pgmap v242: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:51.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:51 smithi026 ceph-mon[27965]: pgmap v243: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:38:51.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:51 smithi169 ceph-mon[31469]: pgmap v243: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:38:51.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:51 smithi190 ceph-mon[31423]: pgmap v243: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:38:53.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:53 smithi026 ceph-mon[27965]: pgmap v244: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:53.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:53 smithi169 ceph-mon[31469]: pgmap v244: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:53.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:53 smithi190 ceph-mon[31423]: pgmap v244: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:55.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:55 smithi026 ceph-mon[27965]: pgmap v245: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:38:55.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:55 smithi169 ceph-mon[31469]: pgmap v245: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:38:55.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:55 smithi190 ceph-mon[31423]: pgmap v245: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:38:57.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:57 smithi026 ceph-mon[27965]: pgmap v246: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:38:57.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:57 smithi169 ceph-mon[31469]: pgmap v246: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:38:57.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:57 smithi190 ceph-mon[31423]: pgmap v246: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:38:59.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:38:59 smithi026 ceph-mon[27965]: pgmap v247: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:59.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:38:59 smithi169 ceph-mon[31469]: pgmap v247: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:38:59.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:38:59 smithi190 ceph-mon[31423]: pgmap v247: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:01.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:01 smithi026 ceph-mon[27965]: pgmap v248: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:39:01.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:01 smithi169 ceph-mon[31469]: pgmap v248: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:39:01.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:01 smithi190 ceph-mon[31423]: pgmap v248: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:39:03.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:03 smithi026 ceph-mon[27965]: pgmap v249: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:03.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:03 smithi169 ceph-mon[31469]: pgmap v249: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:03.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:03 smithi190 ceph-mon[31423]: pgmap v249: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:05.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:05 smithi026 ceph-mon[27965]: pgmap v250: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-21T10:39:05.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:05 smithi169 ceph-mon[31469]: pgmap v250: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-21T10:39:05.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:05 smithi190 ceph-mon[31423]: pgmap v250: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-21T10:39:07.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:07 smithi026 ceph-mon[27965]: pgmap v251: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T10:39:07.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:07 smithi169 ceph-mon[31469]: pgmap v251: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T10:39:07.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:07 smithi190 ceph-mon[31423]: pgmap v251: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T10:39:09.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:09 smithi026 ceph-mon[27965]: pgmap v252: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T10:39:09.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:09 smithi026 ceph-mon[27965]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T10:39:09.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:09 smithi169 ceph-mon[31469]: pgmap v252: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T10:39:09.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:09 smithi169 ceph-mon[31469]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T10:39:09.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:09 smithi190 ceph-mon[31423]: pgmap v252: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T10:39:09.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:09 smithi190 ceph-mon[31423]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T10:39:10.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:10 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:39:10.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:10 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:39:10.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:10 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:39:11.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:11 smithi026 ceph-mon[27965]: pgmap v254: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T10:39:11.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:11 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:39:11.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:11 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:39:11.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:11 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:39:11.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:11 smithi169 ceph-mon[31469]: pgmap v254: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T10:39:11.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:11 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:39:11.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:11 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:39:11.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:11 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:39:11.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:11 smithi190 ceph-mon[31423]: pgmap v254: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T10:39:11.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:11 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:39:11.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:11 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:39:11.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:11 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:39:13.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:13 smithi026 ceph-mon[27965]: pgmap v255: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T10:39:13.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:13 smithi169 ceph-mon[31469]: pgmap v255: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T10:39:13.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:13 smithi190 ceph-mon[31423]: pgmap v255: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-21T10:39:15.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:14 smithi026 ceph-mon[27965]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T10:39:15.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:14 smithi026 ceph-mon[27965]: pgmap v257: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-21T10:39:15.197 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:14 smithi169 ceph-mon[31469]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T10:39:15.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:14 smithi169 ceph-mon[31469]: pgmap v257: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-21T10:39:15.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:14 smithi190 ceph-mon[31423]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T10:39:15.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:14 smithi190 ceph-mon[31423]: pgmap v257: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-21T10:39:17.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:17 smithi190 ceph-mon[31423]: pgmap v258: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:39:17.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:17 smithi026 ceph-mon[27965]: pgmap v258: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:39:17.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:17 smithi169 ceph-mon[31469]: pgmap v258: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:39:19.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:19 smithi190 ceph-mon[31423]: pgmap v259: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-21T10:39:19.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:19 smithi026 ceph-mon[27965]: pgmap v259: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-21T10:39:19.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:19 smithi169 ceph-mon[31469]: pgmap v259: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-04-21T10:39:21.277 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:21 smithi190 ceph-mon[31423]: pgmap v260: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:39:21.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:21 smithi026 ceph-mon[27965]: pgmap v260: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:39:21.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:21 smithi169 ceph-mon[31469]: pgmap v260: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:39:23.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:23 smithi190 ceph-mon[31423]: pgmap v261: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:39:23.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:23 smithi026 ceph-mon[27965]: pgmap v261: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:39:23.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:23 smithi169 ceph-mon[31469]: pgmap v261: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:39:25.198 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:24 smithi169 ceph-mon[31469]: pgmap v262: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-21T10:39:25.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:24 smithi190 ceph-mon[31423]: pgmap v262: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-21T10:39:25.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:24 smithi026 ceph-mon[27965]: pgmap v262: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-21T10:39:27.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:27 smithi190 ceph-mon[31423]: pgmap v263: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T10:39:27.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:27 smithi026 ceph-mon[27965]: pgmap v263: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T10:39:27.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:27 smithi169 ceph-mon[31469]: pgmap v263: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T10:39:29.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:29 smithi190 ceph-mon[31423]: pgmap v264: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:29.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:29 smithi026 ceph-mon[27965]: pgmap v264: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:29.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:29 smithi169 ceph-mon[31469]: pgmap v264: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:31.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:31 smithi190 ceph-mon[31423]: pgmap v265: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:39:31.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:31 smithi026 ceph-mon[27965]: pgmap v265: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:39:31.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:31 smithi169 ceph-mon[31469]: pgmap v265: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:39:33.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:33 smithi190 ceph-mon[31423]: pgmap v266: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:33.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:33 smithi026 ceph-mon[27965]: pgmap v266: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:33.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:33 smithi169 ceph-mon[31469]: pgmap v266: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:35.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:35 smithi190 ceph-mon[31423]: pgmap v267: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:39:35.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:35 smithi026 ceph-mon[27965]: pgmap v267: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:39:35.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:35 smithi169 ceph-mon[31469]: pgmap v267: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:39:37.276 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:37 smithi190 ceph-mon[31423]: pgmap v268: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:39:37.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:37 smithi026 ceph-mon[27965]: pgmap v268: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:39:37.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:37 smithi169 ceph-mon[31469]: pgmap v268: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:39:39.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:39 smithi026 ceph-mon[27965]: pgmap v269: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:39.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:39 smithi169 ceph-mon[31469]: pgmap v269: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:39.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:39 smithi190 ceph-mon[31423]: pgmap v269: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:41.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:41 smithi026 ceph-mon[27965]: pgmap v270: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:39:41.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:41 smithi169 ceph-mon[31469]: pgmap v270: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:39:41.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:41 smithi190 ceph-mon[31423]: pgmap v270: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:39:43.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:43 smithi026 ceph-mon[27965]: pgmap v271: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:43.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:43 smithi169 ceph-mon[31469]: pgmap v271: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:43.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:43 smithi190 ceph-mon[31423]: pgmap v271: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:45.329 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:45 smithi190 ceph-mon[31423]: pgmap v272: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:39:45.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:45 smithi026 ceph-mon[27965]: pgmap v272: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:39:45.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:45 smithi169 ceph-mon[31469]: pgmap v272: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:39:47.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:47 smithi026 ceph-mon[27965]: pgmap v273: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:39:47.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:47 smithi169 ceph-mon[31469]: pgmap v273: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:39:47.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:47 smithi190 ceph-mon[31423]: pgmap v273: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:39:49.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:49 smithi026 ceph-mon[27965]: pgmap v274: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:49.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:49 smithi169 ceph-mon[31469]: pgmap v274: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:49.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:49 smithi190 ceph-mon[31423]: pgmap v274: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:51.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:51 smithi026 ceph-mon[27965]: pgmap v275: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:39:51.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:51 smithi169 ceph-mon[31469]: pgmap v275: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:39:51.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:51 smithi190 ceph-mon[31423]: pgmap v275: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:39:53.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:53 smithi026 ceph-mon[27965]: pgmap v276: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:53.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:53 smithi169 ceph-mon[31469]: pgmap v276: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:53.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:53 smithi190 ceph-mon[31423]: pgmap v276: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:55.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:55 smithi026 ceph-mon[27965]: pgmap v277: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:39:55.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:55 smithi169 ceph-mon[31469]: pgmap v277: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:39:55.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:55 smithi190 ceph-mon[31423]: pgmap v277: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:39:57.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:57 smithi026 ceph-mon[27965]: pgmap v278: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:39:57.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:57 smithi169 ceph-mon[31469]: pgmap v278: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:39:57.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:57 smithi190 ceph-mon[31423]: pgmap v278: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:39:59.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:39:59 smithi026 ceph-mon[27965]: pgmap v279: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:59.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:39:59 smithi169 ceph-mon[31469]: pgmap v279: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:39:59.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:39:59 smithi190 ceph-mon[31423]: pgmap v279: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:00.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:00 smithi026 ceph-mon[27965]: overall HEALTH_OK 2024-04-21T10:40:00.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:00 smithi169 ceph-mon[31469]: overall HEALTH_OK 2024-04-21T10:40:00.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:00 smithi190 ceph-mon[31423]: overall HEALTH_OK 2024-04-21T10:40:01.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:01 smithi026 ceph-mon[27965]: pgmap v280: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:40:01.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:01 smithi169 ceph-mon[31469]: pgmap v280: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:40:01.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:01 smithi190 ceph-mon[31423]: pgmap v280: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:40:03.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:03 smithi026 ceph-mon[27965]: pgmap v281: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:03.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:03 smithi169 ceph-mon[31469]: pgmap v281: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:03.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:03 smithi190 ceph-mon[31423]: pgmap v281: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:05.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:05 smithi026 ceph-mon[27965]: pgmap v282: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-21T10:40:05.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:05 smithi169 ceph-mon[31469]: pgmap v282: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-21T10:40:05.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:05 smithi190 ceph-mon[31423]: pgmap v282: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-21T10:40:07.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:07 smithi026 ceph-mon[27965]: pgmap v283: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T10:40:07.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:07 smithi169 ceph-mon[31469]: pgmap v283: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T10:40:07.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:07 smithi190 ceph-mon[31423]: pgmap v283: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-21T10:40:09.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:09 smithi026 ceph-mon[27965]: pgmap v284: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T10:40:09.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:09 smithi026 ceph-mon[27965]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T10:40:09.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:09 smithi169 ceph-mon[31469]: pgmap v284: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T10:40:09.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:09 smithi169 ceph-mon[31469]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T10:40:09.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:09 smithi190 ceph-mon[31423]: pgmap v284: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-21T10:40:09.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:09 smithi190 ceph-mon[31423]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T10:40:11.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:11 smithi026 ceph-mon[27965]: pgmap v286: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-21T10:40:11.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:11 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:40:11.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:11 smithi169 ceph-mon[31469]: pgmap v286: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-21T10:40:11.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:11 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:40:11.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:11 smithi190 ceph-mon[31423]: pgmap v286: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-21T10:40:11.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:11 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:40:12.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:12 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:40:12.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:12 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:40:12.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:12 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:40:12.447 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:12 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:40:12.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:12 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:40:12.448 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:12 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:40:12.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:12 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:40:12.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:12 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:40:12.527 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:12 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:40:13.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:13 smithi026 ceph-mon[27965]: pgmap v287: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-21T10:40:13.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:13 smithi169 ceph-mon[31469]: pgmap v287: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-21T10:40:13.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:13 smithi190 ceph-mon[31423]: pgmap v287: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-21T10:40:15.579 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:15 smithi190 ceph-mon[31423]: pgmap v288: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-04-21T10:40:15.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:15 smithi026 ceph-mon[27965]: pgmap v288: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-04-21T10:40:15.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:15 smithi169 ceph-mon[31469]: pgmap v288: 97 pgs: 97 active+clean; 65 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-04-21T10:40:17.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:17 smithi026 ceph-mon[27965]: pgmap v289: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-21T10:40:17.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:17 smithi169 ceph-mon[31469]: pgmap v289: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-21T10:40:17.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:17 smithi190 ceph-mon[31423]: pgmap v289: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-21T10:40:19.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:19 smithi026 ceph-mon[27965]: pgmap v290: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-21T10:40:19.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:19 smithi026 ceph-mon[27965]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T10:40:19.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:19 smithi169 ceph-mon[31469]: pgmap v290: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-21T10:40:19.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:19 smithi169 ceph-mon[31469]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T10:40:19.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:19 smithi190 ceph-mon[31423]: pgmap v290: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-21T10:40:19.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:19 smithi190 ceph-mon[31423]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T10:40:21.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:21 smithi026 ceph-mon[27965]: pgmap v292: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:40:21.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:21 smithi169 ceph-mon[31469]: pgmap v292: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:40:21.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:21 smithi190 ceph-mon[31423]: pgmap v292: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:40:23.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:23 smithi026 ceph-mon[27965]: pgmap v293: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:40:23.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:23 smithi169 ceph-mon[31469]: pgmap v293: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:40:23.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:23 smithi190 ceph-mon[31423]: pgmap v293: 97 pgs: 97 active+clean; 65 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:40:25.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:25 smithi026 ceph-mon[27965]: pgmap v294: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:40:25.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:25 smithi169 ceph-mon[31469]: pgmap v294: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:40:25.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:25 smithi190 ceph-mon[31423]: pgmap v294: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:40:27.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:27 smithi026 ceph-mon[27965]: pgmap v295: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-21T10:40:27.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:27 smithi169 ceph-mon[31469]: pgmap v295: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-21T10:40:27.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:27 smithi190 ceph-mon[31423]: pgmap v295: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-21T10:40:29.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:29 smithi026 ceph-mon[27965]: pgmap v296: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-21T10:40:29.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:29 smithi169 ceph-mon[31469]: pgmap v296: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-21T10:40:29.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:29 smithi190 ceph-mon[31423]: pgmap v296: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-21T10:40:31.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:31 smithi026 ceph-mon[27965]: pgmap v297: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2024-04-21T10:40:31.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:31 smithi169 ceph-mon[31469]: pgmap v297: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2024-04-21T10:40:31.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:31 smithi190 ceph-mon[31423]: pgmap v297: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2024-04-21T10:40:33.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:33 smithi026 ceph-mon[27965]: pgmap v298: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:33.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:33 smithi169 ceph-mon[31469]: pgmap v298: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:33.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:33 smithi190 ceph-mon[31423]: pgmap v298: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:35.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:35 smithi026 ceph-mon[27965]: pgmap v299: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:40:35.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:35 smithi169 ceph-mon[31469]: pgmap v299: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:40:35.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:35 smithi190 ceph-mon[31423]: pgmap v299: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:40:37.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:37 smithi026 ceph-mon[27965]: pgmap v300: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:40:37.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:37 smithi169 ceph-mon[31469]: pgmap v300: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:40:37.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:37 smithi190 ceph-mon[31423]: pgmap v300: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:40:39.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:39 smithi026 ceph-mon[27965]: pgmap v301: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:39.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:39 smithi169 ceph-mon[31469]: pgmap v301: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:39.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:39 smithi190 ceph-mon[31423]: pgmap v301: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:41.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:41 smithi026 ceph-mon[27965]: pgmap v302: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:40:41.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:41 smithi169 ceph-mon[31469]: pgmap v302: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:40:41.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:41 smithi190 ceph-mon[31423]: pgmap v302: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:40:43.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:43 smithi026 ceph-mon[27965]: pgmap v303: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:43.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:43 smithi169 ceph-mon[31469]: pgmap v303: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:43.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:43 smithi190 ceph-mon[31423]: pgmap v303: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:45.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:45 smithi026 ceph-mon[27965]: pgmap v304: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:40:45.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:45 smithi169 ceph-mon[31469]: pgmap v304: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:40:45.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:45 smithi190 ceph-mon[31423]: pgmap v304: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:40:47.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:47 smithi026 ceph-mon[27965]: pgmap v305: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:40:47.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:47 smithi169 ceph-mon[31469]: pgmap v305: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:40:47.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:47 smithi190 ceph-mon[31423]: pgmap v305: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:40:49.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:49 smithi026 ceph-mon[27965]: pgmap v306: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:49.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:49 smithi169 ceph-mon[31469]: pgmap v306: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:49.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:49 smithi190 ceph-mon[31423]: pgmap v306: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:40:51.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:51 smithi026 ceph-mon[27965]: pgmap v307: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:40:51.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:51 smithi169 ceph-mon[31469]: pgmap v307: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:40:51.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:51 smithi190 ceph-mon[31423]: pgmap v307: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:40:53.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:53 smithi026 ceph-mon[27965]: pgmap v308: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:40:53.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:53 smithi169 ceph-mon[31469]: pgmap v308: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:40:53.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:53 smithi190 ceph-mon[31423]: pgmap v308: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:40:55.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:55 smithi026 ceph-mon[27965]: pgmap v309: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:40:55.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:55 smithi169 ceph-mon[31469]: pgmap v309: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:40:55.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:55 smithi190 ceph-mon[31423]: pgmap v309: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:40:57.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:57 smithi026 ceph-mon[27965]: pgmap v310: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-21T10:40:57.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:57 smithi169 ceph-mon[31469]: pgmap v310: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-21T10:40:57.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:57 smithi190 ceph-mon[31423]: pgmap v310: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-21T10:40:59.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:40:59 smithi026 ceph-mon[27965]: pgmap v311: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:40:59.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:40:59 smithi169 ceph-mon[31469]: pgmap v311: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:40:59.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:40:59 smithi190 ceph-mon[31423]: pgmap v311: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-21T10:41:00.625 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-21T10:41:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.733+0000 7fe292a9a640 1 -- 172.21.15.26:0/511145276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe28c1010c0 msgr2=0x7fe28c1014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:00.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.733+0000 7fe292a9a640 1 --2- 172.21.15.26:0/511145276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe28c1010c0 0x7fe28c1014a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe288009900 tx=0x7fe28802bc30 comp rx=0 tx=0).stop 2024-04-21T10:41:00.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 -- 172.21.15.26:0/511145276 shutdown_connections 2024-04-21T10:41:00.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 --2- 172.21.15.26:0/511145276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe28c106190 0x7fe28c10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:00.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 --2- 172.21.15.26:0/511145276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe28c101a70 0x7fe28c105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:00.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 --2- 172.21.15.26:0/511145276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe28c1010c0 0x7fe28c1014a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:00.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 -- 172.21.15.26:0/511145276 >> 172.21.15.26:0/511145276 conn(0x7fe28c077fd0 msgr2=0x7fe28c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:00.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 -- 172.21.15.26:0/511145276 shutdown_connections 2024-04-21T10:41:00.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 -- 172.21.15.26:0/511145276 wait complete. 2024-04-21T10:41:00.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 Processor -- start 2024-04-21T10:41:00.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 -- start start 2024-04-21T10:41:00.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe28c1010c0 0x7fe28c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:00.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe28c106190 0x7fe28c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:00.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe28c19cfb0 0x7fe28c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:00.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1110f0 con 0x7fe28c1010c0 2024-04-21T10:41:00.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c110f70 con 0x7fe28c19cfb0 2024-04-21T10:41:00.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.734+0000 7fe292a9a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c111270 con 0x7fe28c106190 2024-04-21T10:41:00.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe290836640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe28c1010c0 0x7fe28c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:00.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe290836640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe28c1010c0 0x7fe28c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34690/0 (socket says 172.21.15.26:34690) 2024-04-21T10:41:00.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe290836640 1 -- 172.21.15.26:0/3112216073 learned_addr learned my addr 172.21.15.26:0/3112216073 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:00.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe283fff640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe28c106190 0x7fe28c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:00.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe290836640 1 -- 172.21.15.26:0/3112216073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe28c106190 msgr2=0x7fe28c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:00.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe290836640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe28c106190 0x7fe28c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:00.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe290836640 1 -- 172.21.15.26:0/3112216073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe28c19cfb0 msgr2=0x7fe28c19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:00.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe291037640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe28c19cfb0 0x7fe28c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:00.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe290836640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe28c19cfb0 0x7fe28c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:00.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe290836640 1 -- 172.21.15.26:0/3112216073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28c19daa0 con 0x7fe28c1010c0 2024-04-21T10:41:00.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe290836640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe28c1010c0 0x7fe28c1986f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe288005940 tx=0x7fe2880045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:00.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe281ffb640 1 -- 172.21.15.26:0/3112216073 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288043020 con 0x7fe28c1010c0 2024-04-21T10:41:00.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe281ffb640 1 -- 172.21.15.26:0/3112216073 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe28802caf0 con 0x7fe28c1010c0 2024-04-21T10:41:00.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe292a9a640 1 -- 172.21.15.26:0/3112216073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28c19dd30 con 0x7fe28c1010c0 2024-04-21T10:41:00.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.735+0000 7fe281ffb640 1 -- 172.21.15.26:0/3112216073 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288003af0 con 0x7fe28c1010c0 2024-04-21T10:41:00.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.736+0000 7fe292a9a640 1 -- 172.21.15.26:0/3112216073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28c1a5580 con 0x7fe28c1010c0 2024-04-21T10:41:00.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.736+0000 7fe292a9a640 1 -- 172.21.15.26:0/3112216073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe28c04fce0 con 0x7fe28c1010c0 2024-04-21T10:41:00.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.738+0000 7fe281ffb640 1 -- 172.21.15.26:0/3112216073 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fe288046020 con 0x7fe28c1010c0 2024-04-21T10:41:00.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.738+0000 7fe281ffb640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fe2740a3e40 0x7fe2740a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:00.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.738+0000 7fe281ffb640 1 -- 172.21.15.26:0/3112216073 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7fe2880f2fb0 con 0x7fe28c1010c0 2024-04-21T10:41:00.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.738+0000 7fe283fff640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fe2740a3e40 0x7fe2740a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:00.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.739+0000 7fe283fff640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fe2740a3e40 0x7fe2740a6300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe28c105c10 tx=0x7fe27c005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:00.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.742+0000 7fe281ffb640 1 -- 172.21.15.26:0/3112216073 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe28804d020 con 0x7fe28c1010c0 2024-04-21T10:41:00.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.952+0000 7fe292a9a640 1 -- 172.21.15.26:0/3112216073 --> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fe28c101a70 con 0x7fe2740a3e40 2024-04-21T10:41:00.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.960+0000 7fe281ffb640 1 -- 172.21.15.26:0/3112216073 <== mgr.24869 v2:172.21.15.26:6800/4245046040 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fe28c101a70 con 0x7fe2740a3e40 2024-04-21T10:41:00.959 INFO:teuthology.orchestra.run.smithi026.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-21T00:00:00", "created": "2024-04-21T10:30:59", "first": "2024-04-21T10:31:00", "last": "2024-04-21T10:41:00", "last_pruned": "2024-04-21T10:41:00", "created_count": 11, "pruned_count": 5, "active": true} 2024-04-21T10:41:00.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.962+0000 7fe292a9a640 1 -- 172.21.15.26:0/3112216073 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fe2740a3e40 msgr2=0x7fe2740a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.962+0000 7fe292a9a640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fe2740a3e40 0x7fe2740a6300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe28c105c10 tx=0x7fe27c005f90 comp rx=0 tx=0).stop 2024-04-21T10:41:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.962+0000 7fe292a9a640 1 -- 172.21.15.26:0/3112216073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe28c1010c0 msgr2=0x7fe28c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.962+0000 7fe292a9a640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe28c1010c0 0x7fe28c1986f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe288005940 tx=0x7fe2880045e0 comp rx=0 tx=0).stop 2024-04-21T10:41:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.962+0000 7fe292a9a640 1 -- 172.21.15.26:0/3112216073 shutdown_connections 2024-04-21T10:41:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.962+0000 7fe292a9a640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fe2740a3e40 0x7fe2740a6300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.962+0000 7fe292a9a640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe28c19cfb0 0x7fe28c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.962+0000 7fe292a9a640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe28c106190 0x7fe28c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.962+0000 7fe292a9a640 1 --2- 172.21.15.26:0/3112216073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe28c1010c0 0x7fe28c1986f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.962+0000 7fe292a9a640 1 -- 172.21.15.26:0/3112216073 >> 172.21.15.26:0/3112216073 conn(0x7fe28c077fd0 msgr2=0x7fe28c06ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.962+0000 7fe292a9a640 1 -- 172.21.15.26:0/3112216073 shutdown_connections 2024-04-21T10:41:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:00.962+0000 7fe292a9a640 1 -- 172.21.15.26:0/3112216073 wait complete. 2024-04-21T10:41:00.972 DEBUG:teuthology.orchestra.run.smithi026:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-21T10:41:01.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.077+0000 7f45c9e99640 1 -- 172.21.15.26:0/990264209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c410c520 msgr2=0x7f45c410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.077+0000 7f45c9e99640 1 --2- 172.21.15.26:0/990264209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c410c520 0x7f45c410e9c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009ca0 tx=0x7f45bc02c230 comp rx=0 tx=0).stop 2024-04-21T10:41:01.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.077+0000 7f45c9e99640 1 -- 172.21.15.26:0/990264209 shutdown_connections 2024-04-21T10:41:01.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.077+0000 7f45c9e99640 1 --2- 172.21.15.26:0/990264209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c410c520 0x7f45c410e9c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.077+0000 7f45c9e99640 1 --2- 172.21.15.26:0/990264209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c4103b00 0x7f45c410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.077+0000 7f45c9e99640 1 --2- 172.21.15.26:0/990264209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45c41031e0 0x7f45c41035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.077+0000 7f45c9e99640 1 -- 172.21.15.26:0/990264209 >> 172.21.15.26:0/990264209 conn(0x7f45c4077fd0 msgr2=0x7f45c40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:01.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.077+0000 7f45c9e99640 1 -- 172.21.15.26:0/990264209 shutdown_connections 2024-04-21T10:41:01.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.077+0000 7f45c9e99640 1 -- 172.21.15.26:0/990264209 wait complete. 2024-04-21T10:41:01.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.077+0000 7f45c9e99640 1 Processor -- start 2024-04-21T10:41:01.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c9e99640 1 -- start start 2024-04-21T10:41:01.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c9e99640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45c41031e0 0x7f45c41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:01.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c9e99640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c4103b00 0x7f45c4198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:01.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c9e99640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c410c520 0x7f45c419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:01.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c9e99640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f45c41121b0 con 0x7f45c4103b00 2024-04-21T10:41:01.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c9e99640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45c4112030 con 0x7f45c41031e0 2024-04-21T10:41:01.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c9e99640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f45c4112330 con 0x7f45c410c520 2024-04-21T10:41:01.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c4103b00 0x7f45c4198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:01.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c4103b00 0x7f45c4198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34720/0 (socket says 172.21.15.26:34720) 2024-04-21T10:41:01.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c2ffd640 1 -- 172.21.15.26:0/1882882598 learned_addr learned my addr 172.21.15.26:0/1882882598 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:01.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c37fe640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45c41031e0 0x7f45c41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:01.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c3fff640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c410c520 0x7f45c419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:01.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c2ffd640 1 -- 172.21.15.26:0/1882882598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c410c520 msgr2=0x7f45c419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c2ffd640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c410c520 0x7f45c419cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c2ffd640 1 -- 172.21.15.26:0/1882882598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45c41031e0 msgr2=0x7f45c41986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c2ffd640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45c41031e0 0x7f45c41986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c2ffd640 1 -- 172.21.15.26:0/1882882598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c419d650 con 0x7f45c4103b00 2024-04-21T10:41:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.078+0000 7f45c3fff640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c410c520 0x7f45c419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:41:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.079+0000 7f45c2ffd640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c4103b00 0x7f45c4198bf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f45b800d770 tx=0x7f45b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.079+0000 7f45c0ff9640 1 -- 172.21.15.26:0/1882882598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8017070 con 0x7f45c4103b00 2024-04-21T10:41:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.079+0000 7f45c0ff9640 1 -- 172.21.15.26:0/1882882598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45b80054d0 con 0x7f45c4103b00 2024-04-21T10:41:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.079+0000 7f45c9e99640 1 -- 172.21.15.26:0/1882882598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c419d940 con 0x7f45c4103b00 2024-04-21T10:41:01.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.079+0000 7f45c0ff9640 1 -- 172.21.15.26:0/1882882598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8004c30 con 0x7f45c4103b00 2024-04-21T10:41:01.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.079+0000 7f45c9e99640 1 -- 172.21.15.26:0/1882882598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c41a51d0 con 0x7f45c4103b00 2024-04-21T10:41:01.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.080+0000 7f45c9e99640 1 -- 172.21.15.26:0/1882882598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c404fce0 con 0x7f45c4103b00 2024-04-21T10:41:01.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.081+0000 7f45c0ff9640 1 -- 172.21.15.26:0/1882882598 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f45b8004dd0 con 0x7f45c4103b00 2024-04-21T10:41:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.081+0000 7f45c0ff9640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f45ac0a3e40 0x7f45ac0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.082+0000 7f45c0ff9640 1 -- 172.21.15.26:0/1882882598 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7f45b80d05e0 con 0x7f45c4103b00 2024-04-21T10:41:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.082+0000 7f45c37fe640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f45ac0a3e40 0x7f45ac0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.082+0000 7f45c37fe640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f45ac0a3e40 0x7f45ac0a6300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005d00 tx=0x7f45b400b410 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.085+0000 7f45c0ff9640 1 -- 172.21.15.26:0/1882882598 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45b809cce0 con 0x7f45c4103b00 2024-04-21T10:41:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.293+0000 7f45c9e99640 1 -- 172.21.15.26:0/1882882598 --> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f45c4062390 con 0x7f45ac0a3e40 2024-04-21T10:41:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.300+0000 7f45c0ff9640 1 -- 172.21.15.26:0/1882882598 <== mgr.24869 v2:172.21.15.26:6800/4245046040 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f45c4062390 con 0x7f45ac0a3e40 2024-04-21T10:41:01.301 INFO:teuthology.orchestra.run.smithi026.stdout:/ 1m 6m3h 2024-04-21T10:41:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.303+0000 7f45c9e99640 1 -- 172.21.15.26:0/1882882598 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f45ac0a3e40 msgr2=0x7f45ac0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.303+0000 7f45c9e99640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f45ac0a3e40 0x7f45ac0a6300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005d00 tx=0x7f45b400b410 comp rx=0 tx=0).stop 2024-04-21T10:41:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.303+0000 7f45c9e99640 1 -- 172.21.15.26:0/1882882598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c4103b00 msgr2=0x7f45c4198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.303+0000 7f45c9e99640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c4103b00 0x7f45c4198bf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f45b800d770 tx=0x7f45b800dc30 comp rx=0 tx=0).stop 2024-04-21T10:41:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.303+0000 7f45c9e99640 1 -- 172.21.15.26:0/1882882598 shutdown_connections 2024-04-21T10:41:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.303+0000 7f45c9e99640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f45ac0a3e40 0x7f45ac0a6300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.303+0000 7f45c9e99640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c410c520 0x7f45c419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.303+0000 7f45c9e99640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c4103b00 0x7f45c4198bf0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.303+0000 7f45c9e99640 1 --2- 172.21.15.26:0/1882882598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45c41031e0 0x7f45c41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.303+0000 7f45c9e99640 1 -- 172.21.15.26:0/1882882598 >> 172.21.15.26:0/1882882598 conn(0x7f45c4077fd0 msgr2=0x7f45c40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.303+0000 7f45c9e99640 1 -- 172.21.15.26:0/1882882598 shutdown_connections 2024-04-21T10:41:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.303+0000 7f45c9e99640 1 -- 172.21.15.26:0/1882882598 wait complete. 2024-04-21T10:41:01.313 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-21T10:41:01.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:41:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.428+0000 7f5081893640 1 -- 172.21.15.26:0/3307104672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f507c105e30 msgr2=0x7f507c106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.428+0000 7f5081893640 1 --2- 172.21.15.26:0/3307104672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f507c105e30 0x7f507c106210 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f506c009cd0 tx=0x7f506c02c0f0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.428+0000 7f5081893640 1 -- 172.21.15.26:0/3307104672 shutdown_connections 2024-04-21T10:41:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.428+0000 7f5081893640 1 --2- 172.21.15.26:0/3307104672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c101b60 0x7f507c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.428+0000 7f5081893640 1 --2- 172.21.15.26:0/3307104672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507c1011a0 0x7f507c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.428+0000 7f5081893640 1 --2- 172.21.15.26:0/3307104672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f507c105e30 0x7f507c106210 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.428+0000 7f5081893640 1 -- 172.21.15.26:0/3307104672 >> 172.21.15.26:0/3307104672 conn(0x7f507c077e30 msgr2=0x7f507c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.428+0000 7f5081893640 1 -- 172.21.15.26:0/3307104672 shutdown_connections 2024-04-21T10:41:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.428+0000 7f5081893640 1 -- 172.21.15.26:0/3307104672 wait complete. 2024-04-21T10:41:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.428+0000 7f5081893640 1 Processor -- start 2024-04-21T10:41:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.428+0000 7f5081893640 1 -- start start 2024-04-21T10:41:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f5081893640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c1011a0 0x7f507c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f5081893640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507c101b60 0x7f507c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f5081893640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f507c105e30 0x7f507c19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f5081893640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f507c1110c0 con 0x7f507c105e30 2024-04-21T10:41:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f5081893640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f507c110f40 con 0x7f507c101b60 2024-04-21T10:41:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f5081893640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f507c111240 con 0x7f507c1011a0 2024-04-21T10:41:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f507c105e30 0x7f507c19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f507c105e30 0x7f507c19cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34742/0 (socket says 172.21.15.26:34742) 2024-04-21T10:41:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507b7fe640 1 -- 172.21.15.26:0/4192840952 learned_addr learned my addr 172.21.15.26:0/4192840952 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507a7fc640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507c101b60 0x7f507c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507affd640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c1011a0 0x7f507c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507b7fe640 1 -- 172.21.15.26:0/4192840952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c1011a0 msgr2=0x7f507c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507b7fe640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c1011a0 0x7f507c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507b7fe640 1 -- 172.21.15.26:0/4192840952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507c101b60 msgr2=0x7f507c198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507b7fe640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507c101b60 0x7f507c198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507b7fe640 1 -- 172.21.15.26:0/4192840952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507c19d6a0 con 0x7f507c105e30 2024-04-21T10:41:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507affd640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c1011a0 0x7f507c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:41:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507a7fc640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507c101b60 0x7f507c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:41:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f507b7fe640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f507c105e30 0x7f507c19cf70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f507400c970 tx=0x7f507400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:01.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.429+0000 7f5080891640 1 -- 172.21.15.26:0/4192840952 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074007bb0 con 0x7f507c105e30 2024-04-21T10:41:01.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.430+0000 7f5080891640 1 -- 172.21.15.26:0/4192840952 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5074007d50 con 0x7f507c105e30 2024-04-21T10:41:01.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.430+0000 7f5081893640 1 -- 172.21.15.26:0/4192840952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507c19d990 con 0x7f507c105e30 2024-04-21T10:41:01.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.430+0000 7f5080891640 1 -- 172.21.15.26:0/4192840952 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074018770 con 0x7f507c105e30 2024-04-21T10:41:01.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.430+0000 7f5081893640 1 -- 172.21.15.26:0/4192840952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507c1a5220 con 0x7f507c105e30 2024-04-21T10:41:01.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.431+0000 7f5081893640 1 -- 172.21.15.26:0/4192840952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f507c102570 con 0x7f507c105e30 2024-04-21T10:41:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.432+0000 7f5080891640 1 -- 172.21.15.26:0/4192840952 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f507400f990 con 0x7f507c105e30 2024-04-21T10:41:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.432+0000 7f5080891640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f50640a3d20 0x7f50640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:01.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.432+0000 7f507affd640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f50640a3d20 0x7f50640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:01.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.432+0000 7f5080891640 1 -- 172.21.15.26:0/4192840952 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7f50740d0970 con 0x7f507c105e30 2024-04-21T10:41:01.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.433+0000 7f507affd640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f50640a3d20 0x7f50640a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f506c006010 tx=0x7f506c005d80 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:01.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.437+0000 7f5080891640 1 -- 172.21.15.26:0/4192840952 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5074014030 con 0x7f507c105e30 2024-04-21T10:41:01.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.627+0000 7f5081893640 1 -- 172.21.15.26:0/4192840952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f507c068de0 con 0x7f507c105e30 2024-04-21T10:41:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.628+0000 7f5080891640 1 -- 172.21.15.26:0/4192840952 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+23169 (secure 0 0 0) 0x7f507c068de0 con 0x7f507c105e30 2024-04-21T10:41:01.628 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:41:01.628 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":98,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:40:18.947413+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3802401516":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/3125048023":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4245538524":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2797499963":"2024-04-22T10:30:57.800969+0000","172.21.15.26:6801/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:41:01.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.630+0000 7f5081893640 1 -- 172.21.15.26:0/4192840952 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f50640a3d20 msgr2=0x7f50640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.630+0000 7f5081893640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f50640a3d20 0x7f50640a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f506c006010 tx=0x7f506c005d80 comp rx=0 tx=0).stop 2024-04-21T10:41:01.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.630+0000 7f5081893640 1 -- 172.21.15.26:0/4192840952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f507c105e30 msgr2=0x7f507c19cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.630+0000 7f5081893640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f507c105e30 0x7f507c19cf70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f507400c970 tx=0x7f507400ce30 comp rx=0 tx=0).stop 2024-04-21T10:41:01.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.630+0000 7f5081893640 1 -- 172.21.15.26:0/4192840952 shutdown_connections 2024-04-21T10:41:01.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.630+0000 7f5081893640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f50640a3d20 0x7f50640a61e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.630+0000 7f5081893640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f507c105e30 0x7f507c19cf70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.630+0000 7f5081893640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507c101b60 0x7f507c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.630+0000 7f5081893640 1 --2- 172.21.15.26:0/4192840952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f507c1011a0 0x7f507c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.630+0000 7f5081893640 1 -- 172.21.15.26:0/4192840952 >> 172.21.15.26:0/4192840952 conn(0x7f507c077e30 msgr2=0x7f507c104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.630+0000 7f5081893640 1 -- 172.21.15.26:0/4192840952 shutdown_connections 2024-04-21T10:41:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.630+0000 7f5081893640 1 -- 172.21.15.26:0/4192840952 wait complete. 2024-04-21T10:41:01.639 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:01 smithi026 ceph-mon[27965]: pgmap v312: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:41:01.641 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T10:24:28.176891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T10:28:59.417687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T10:29:00.234808+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T10:41:01.641 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T10:41:01.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:01 smithi169 ceph-mon[31469]: pgmap v312: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:41:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.747+0000 7fd4bb48c640 1 -- 172.21.15.26:0/4240802121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b4102b40 msgr2=0x7fd4b4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.747+0000 7fd4bb48c640 1 --2- 172.21.15.26:0/4240802121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b4102b40 0x7fd4b4106b80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009900 tx=0x7fd4a402bc30 comp rx=0 tx=0).stop 2024-04-21T10:41:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.747+0000 7fd4bb48c640 1 -- 172.21.15.26:0/4240802121 shutdown_connections 2024-04-21T10:41:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.747+0000 7fd4bb48c640 1 --2- 172.21.15.26:0/4240802121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4b4107260 0x7fd4b410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.747+0000 7fd4bb48c640 1 --2- 172.21.15.26:0/4240802121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b4102b40 0x7fd4b4106b80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.747+0000 7fd4bb48c640 1 --2- 172.21.15.26:0/4240802121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4b4102190 0x7fd4b4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.747+0000 7fd4bb48c640 1 -- 172.21.15.26:0/4240802121 >> 172.21.15.26:0/4240802121 conn(0x7fd4b40fd990 msgr2=0x7fd4b40ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.747+0000 7fd4bb48c640 1 -- 172.21.15.26:0/4240802121 shutdown_connections 2024-04-21T10:41:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.747+0000 7fd4bb48c640 1 -- 172.21.15.26:0/4240802121 wait complete. 2024-04-21T10:41:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4bb48c640 1 Processor -- start 2024-04-21T10:41:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4bb48c640 1 -- start start 2024-04-21T10:41:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4bb48c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4b4102190 0x7fd4b4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4bb48c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b4107260 0x7fd4b4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4bb48c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4b4074cd0 0x7fd4b41a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4bb48c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4112180 con 0x7fd4b4107260 2024-04-21T10:41:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4bb48c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4112000 con 0x7fd4b4102190 2024-04-21T10:41:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4bb48c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4112300 con 0x7fd4b4074cd0 2024-04-21T10:41:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4b8a27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b4107260 0x7fd4b4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4b8a27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b4107260 0x7fd4b4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34770/0 (socket says 172.21.15.26:34770) 2024-04-21T10:41:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4b8a27640 1 -- 172.21.15.26:0/1916762274 learned_addr learned my addr 172.21.15.26:0/1916762274 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4b9228640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4b4102190 0x7fd4b4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.748+0000 7fd4b9a29640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4b4074cd0 0x7fd4b41a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.749+0000 7fd4b8a27640 1 -- 172.21.15.26:0/1916762274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4b4074cd0 msgr2=0x7fd4b41a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.749+0000 7fd4b8a27640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4b4074cd0 0x7fd4b41a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.749+0000 7fd4b8a27640 1 -- 172.21.15.26:0/1916762274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4b4102190 msgr2=0x7fd4b4074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.749+0000 7fd4b8a27640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4b4102190 0x7fd4b4074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.749+0000 7fd4b8a27640 1 -- 172.21.15.26:0/1916762274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b41a7030 con 0x7fd4b4107260 2024-04-21T10:41:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.749+0000 7fd4b8a27640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b4107260 0x7fd4b4074790 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4036040 tx=0x7fd4a402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.749+0000 7fd4aa7fc640 1 -- 172.21.15.26:0/1916762274 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4043020 con 0x7fd4b4107260 2024-04-21T10:41:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.749+0000 7fd4aa7fc640 1 -- 172.21.15.26:0/1916762274 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a402cbe0 con 0x7fd4b4107260 2024-04-21T10:41:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.749+0000 7fd4bb48c640 1 -- 172.21.15.26:0/1916762274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b41a72c0 con 0x7fd4b4107260 2024-04-21T10:41:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.749+0000 7fd4aa7fc640 1 -- 172.21.15.26:0/1916762274 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4003bf0 con 0x7fd4b4107260 2024-04-21T10:41:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.749+0000 7fd4bb48c640 1 -- 172.21.15.26:0/1916762274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b41a7810 con 0x7fd4b4107260 2024-04-21T10:41:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.750+0000 7fd4bb48c640 1 -- 172.21.15.26:0/1916762274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b40691c0 con 0x7fd4b4107260 2024-04-21T10:41:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.751+0000 7fd4aa7fc640 1 -- 172.21.15.26:0/1916762274 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fd4a4046020 con 0x7fd4b4107260 2024-04-21T10:41:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.752+0000 7fd4aa7fc640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4a00a3d50 0x7fd4a00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.752+0000 7fd4aa7fc640 1 -- 172.21.15.26:0/1916762274 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7fd4a40f2f00 con 0x7fd4b4107260 2024-04-21T10:41:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.752+0000 7fd4b9228640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4a00a3d50 0x7fd4a00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.752+0000 7fd4b9228640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4a00a3d50 0x7fd4a00a6210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000bbe0 tx=0x7fd4b000b390 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.755+0000 7fd4aa7fc640 1 -- 172.21.15.26:0/1916762274 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4a404d020 con 0x7fd4b4107260 2024-04-21T10:41:01.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:01 smithi190 ceph-mon[31423]: pgmap v312: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:41:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.941+0000 7fd4bb48c640 1 -- 172.21.15.26:0/1916762274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd4b410a490 con 0x7fd4b4107260 2024-04-21T10:41:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.942+0000 7fd4aa7fc640 1 -- 172.21.15.26:0/1916762274 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7fd4a40bf690 con 0x7fd4b4107260 2024-04-21T10:41:01.941 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-21T10:41:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.944+0000 7fd4bb48c640 1 -- 172.21.15.26:0/1916762274 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4a00a3d50 msgr2=0x7fd4a00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.944+0000 7fd4bb48c640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4a00a3d50 0x7fd4a00a6210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000bbe0 tx=0x7fd4b000b390 comp rx=0 tx=0).stop 2024-04-21T10:41:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.944+0000 7fd4bb48c640 1 -- 172.21.15.26:0/1916762274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b4107260 msgr2=0x7fd4b4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.944+0000 7fd4bb48c640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b4107260 0x7fd4b4074790 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4036040 tx=0x7fd4a402c7e0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.944+0000 7fd4bb48c640 1 -- 172.21.15.26:0/1916762274 shutdown_connections 2024-04-21T10:41:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.944+0000 7fd4bb48c640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd4a00a3d50 0x7fd4a00a6210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.944+0000 7fd4bb48c640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4b4074cd0 0x7fd4b41a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.944+0000 7fd4bb48c640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b4107260 0x7fd4b4074790 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.944+0000 7fd4bb48c640 1 --2- 172.21.15.26:0/1916762274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4b4102190 0x7fd4b4074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.944+0000 7fd4bb48c640 1 -- 172.21.15.26:0/1916762274 >> 172.21.15.26:0/1916762274 conn(0x7fd4b40fd990 msgr2=0x7fd4b40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.944+0000 7fd4bb48c640 1 -- 172.21.15.26:0/1916762274 shutdown_connections 2024-04-21T10:41:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:01.944+0000 7fd4bb48c640 1 -- 172.21.15.26:0/1916762274 wait complete. 2024-04-21T10:41:01.954 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T10:41:02.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.061+0000 7fd991881640 1 -- 172.21.15.26:0/1900724224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98c102190 msgr2=0x7fd98c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.061+0000 7fd991881640 1 --2- 172.21.15.26:0/1900724224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98c102190 0x7fd98c102570 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009900 tx=0x7fd97c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:41:02.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.061+0000 7fd991881640 1 -- 172.21.15.26:0/1900724224 shutdown_connections 2024-04-21T10:41:02.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.061+0000 7fd991881640 1 --2- 172.21.15.26:0/1900724224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98c107260 0x7fd98c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.061+0000 7fd991881640 1 --2- 172.21.15.26:0/1900724224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd98c102b40 0x7fd98c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.061+0000 7fd991881640 1 --2- 172.21.15.26:0/1900724224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98c102190 0x7fd98c102570 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.061+0000 7fd991881640 1 -- 172.21.15.26:0/1900724224 >> 172.21.15.26:0/1900724224 conn(0x7fd98c0fd990 msgr2=0x7fd98c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:02.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.061+0000 7fd991881640 1 -- 172.21.15.26:0/1900724224 shutdown_connections 2024-04-21T10:41:02.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.061+0000 7fd991881640 1 -- 172.21.15.26:0/1900724224 wait complete. 2024-04-21T10:41:02.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.062+0000 7fd991881640 1 Processor -- start 2024-04-21T10:41:02.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.062+0000 7fd991881640 1 -- start start 2024-04-21T10:41:02.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.062+0000 7fd991881640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98c102190 0x7fd98c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:02.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.062+0000 7fd991881640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd98c107260 0x7fd98c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:02.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.062+0000 7fd991881640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98c198640 0x7fd98c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:02.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.062+0000 7fd991881640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c112240 con 0x7fd98c102190 2024-04-21T10:41:02.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.062+0000 7fd991881640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c1120c0 con 0x7fd98c198640 2024-04-21T10:41:02.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.062+0000 7fd991881640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c1123c0 con 0x7fd98c107260 2024-04-21T10:41:02.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.062+0000 7fd98affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98c102190 0x7fd98c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:02.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.062+0000 7fd98affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98c102190 0x7fd98c19e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34784/0 (socket says 172.21.15.26:34784) 2024-04-21T10:41:02.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.062+0000 7fd98affd640 1 -- 172.21.15.26:0/886070095 learned_addr learned my addr 172.21.15.26:0/886070095 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:02.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.062+0000 7fd98b7fe640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98c198640 0x7fd98c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:02.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.063+0000 7fd98affd640 1 -- 172.21.15.26:0/886070095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd98c107260 msgr2=0x7fd98c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.063+0000 7fd98affd640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd98c107260 0x7fd98c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.063+0000 7fd98affd640 1 -- 172.21.15.26:0/886070095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98c198640 msgr2=0x7fd98c198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.063+0000 7fd98affd640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98c198640 0x7fd98c198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.063+0000 7fd98affd640 1 -- 172.21.15.26:0/886070095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98c199240 con 0x7fd98c102190 2024-04-21T10:41:02.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.063+0000 7fd98b7fe640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98c198640 0x7fd98c198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:41:02.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.063+0000 7fd98affd640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98c102190 0x7fd98c19e580 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd97c005940 tx=0x7fd97c02c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:02.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.063+0000 7fd99087f640 1 -- 172.21.15.26:0/886070095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c043020 con 0x7fd98c102190 2024-04-21T10:41:02.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.063+0000 7fd991881640 1 -- 172.21.15.26:0/886070095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98c1994d0 con 0x7fd98c102190 2024-04-21T10:41:02.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.063+0000 7fd99087f640 1 -- 172.21.15.26:0/886070095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd97c02cc60 con 0x7fd98c102190 2024-04-21T10:41:02.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.063+0000 7fd99087f640 1 -- 172.21.15.26:0/886070095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c003c30 con 0x7fd98c102190 2024-04-21T10:41:02.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.063+0000 7fd991881640 1 -- 172.21.15.26:0/886070095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd98c1a5670 con 0x7fd98c102190 2024-04-21T10:41:02.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.064+0000 7fd991881640 1 -- 172.21.15.26:0/886070095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd98c103660 con 0x7fd98c102190 2024-04-21T10:41:02.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.065+0000 7fd99087f640 1 -- 172.21.15.26:0/886070095 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fd97c046020 con 0x7fd98c102190 2024-04-21T10:41:02.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.066+0000 7fd99087f640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd9740a3d20 0x7fd9740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:02.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.066+0000 7fd98a7fc640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd9740a3d20 0x7fd9740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:02.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.066+0000 7fd99087f640 1 -- 172.21.15.26:0/886070095 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7fd97c0f2c30 con 0x7fd98c102190 2024-04-21T10:41:02.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.066+0000 7fd98a7fc640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd9740a3d20 0x7fd9740a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd98c199e90 tx=0x7fd980005bf0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:02.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.070+0000 7fd99087f640 1 -- 172.21.15.26:0/886070095 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd97c04d020 con 0x7fd98c102190 2024-04-21T10:41:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.254+0000 7fd991881640 1 -- 172.21.15.26:0/886070095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd98c199b70 con 0x7fd98c102190 2024-04-21T10:41:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.255+0000 7fd99087f640 1 -- 172.21.15.26:0/886070095 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fd97c0dde20 con 0x7fd98c102190 2024-04-21T10:41:02.254 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-21T10:41:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.256+0000 7fd991881640 1 -- 172.21.15.26:0/886070095 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd9740a3d20 msgr2=0x7fd9740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.256+0000 7fd991881640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd9740a3d20 0x7fd9740a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd98c199e90 tx=0x7fd980005bf0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.256+0000 7fd991881640 1 -- 172.21.15.26:0/886070095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98c102190 msgr2=0x7fd98c19e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.256+0000 7fd991881640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98c102190 0x7fd98c19e580 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd97c005940 tx=0x7fd97c02c860 comp rx=0 tx=0).stop 2024-04-21T10:41:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.257+0000 7fd991881640 1 -- 172.21.15.26:0/886070095 shutdown_connections 2024-04-21T10:41:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.257+0000 7fd991881640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fd9740a3d20 0x7fd9740a61e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.257+0000 7fd991881640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd98c198640 0x7fd98c198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.257+0000 7fd991881640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd98c107260 0x7fd98c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.257+0000 7fd991881640 1 --2- 172.21.15.26:0/886070095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd98c102190 0x7fd98c19e580 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.257+0000 7fd991881640 1 -- 172.21.15.26:0/886070095 >> 172.21.15.26:0/886070095 conn(0x7fd98c0fd990 msgr2=0x7fd98c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.257+0000 7fd991881640 1 -- 172.21.15.26:0/886070095 shutdown_connections 2024-04-21T10:41:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.257+0000 7fd991881640 1 -- 172.21.15.26:0/886070095 wait complete. 2024-04-21T10:41:02.267 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T10:41:02.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.373+0000 7f0943db2640 1 -- 172.21.15.26:0/1146443599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f093c107270 msgr2=0x7f093c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.373+0000 7f0943db2640 1 --2- 172.21.15.26:0/1146443599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f093c107270 0x7f093c10ee40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0934009ca0 tx=0x7f093402c0f0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.373+0000 7f0943db2640 1 -- 172.21.15.26:0/1146443599 shutdown_connections 2024-04-21T10:41:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.373+0000 7f0943db2640 1 --2- 172.21.15.26:0/1146443599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f093c107270 0x7f093c10ee40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.373+0000 7f0943db2640 1 --2- 172.21.15.26:0/1146443599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f093c06b1f0 0x7f093c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.373+0000 7f0943db2640 1 --2- 172.21.15.26:0/1146443599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f093c06a840 0x7f093c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.373+0000 7f0943db2640 1 -- 172.21.15.26:0/1146443599 >> 172.21.15.26:0/1146443599 conn(0x7f093c077eb0 msgr2=0x7f093c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.373+0000 7f0943db2640 1 -- 172.21.15.26:0/1146443599 shutdown_connections 2024-04-21T10:41:02.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.374+0000 7f0943db2640 1 -- 172.21.15.26:0/1146443599 wait complete. 2024-04-21T10:41:02.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.374+0000 7f0943db2640 1 Processor -- start 2024-04-21T10:41:02.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.374+0000 7f0943db2640 1 -- start start 2024-04-21T10:41:02.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0943db2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f093c06a840 0x7f093c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:02.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0943db2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f093c107270 0x7f093c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:02.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0943db2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f093c19cf60 0x7f093c19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:02.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0943db2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f093c1121d0 con 0x7f093c06a840 2024-04-21T10:41:02.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0943db2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f093c112050 con 0x7f093c107270 2024-04-21T10:41:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0943db2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f093c112350 con 0x7f093c19cf60 2024-04-21T10:41:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0941b4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f093c06a840 0x7f093c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0941b4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f093c06a840 0x7f093c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34804/0 (socket says 172.21.15.26:34804) 2024-04-21T10:41:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0941b4e640 1 -- 172.21.15.26:0/1528130518 learned_addr learned my addr 172.21.15.26:0/1528130518 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f094134d640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f093c107270 0x7f093c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f094234f640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f093c19cf60 0x7f093c19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0941b4e640 1 -- 172.21.15.26:0/1528130518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f093c19cf60 msgr2=0x7f093c19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0941b4e640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f093c19cf60 0x7f093c19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0941b4e640 1 -- 172.21.15.26:0/1528130518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f093c107270 msgr2=0x7f093c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0941b4e640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f093c107270 0x7f093c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0941b4e640 1 -- 172.21.15.26:0/1528130518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c19da50 con 0x7f093c06a840 2024-04-21T10:41:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f094134d640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f093c107270 0x7f093c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:41:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.375+0000 7f0941b4e640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f093c06a840 0x7f093c1986f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f093800e9f0 tx=0x7f093800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.376+0000 7f0932ffd640 1 -- 172.21.15.26:0/1528130518 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093800cd80 con 0x7f093c06a840 2024-04-21T10:41:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.376+0000 7f0932ffd640 1 -- 172.21.15.26:0/1528130518 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09380108e0 con 0x7f093c06a840 2024-04-21T10:41:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.376+0000 7f0943db2640 1 -- 172.21.15.26:0/1528130518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093c19dd40 con 0x7f093c06a840 2024-04-21T10:41:02.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.376+0000 7f0932ffd640 1 -- 172.21.15.26:0/1528130518 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938018980 con 0x7f093c06a840 2024-04-21T10:41:02.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.377+0000 7f0943db2640 1 -- 172.21.15.26:0/1528130518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093c1a5580 con 0x7f093c06a840 2024-04-21T10:41:02.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.378+0000 7f0943db2640 1 -- 172.21.15.26:0/1528130518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f093c0691c0 con 0x7f093c06a840 2024-04-21T10:41:02.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.379+0000 7f0932ffd640 1 -- 172.21.15.26:0/1528130518 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0938010430 con 0x7f093c06a840 2024-04-21T10:41:02.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.379+0000 7f0932ffd640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f09280a3da0 0x7f09280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:02.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.379+0000 7f094134d640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f09280a3da0 0x7f09280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:02.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.380+0000 7f0932ffd640 1 -- 172.21.15.26:0/1528130518 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7f09380d0c50 con 0x7f093c06a840 2024-04-21T10:41:02.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.380+0000 7f094134d640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f09280a3da0 0x7f09280a6260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f093c108710 tx=0x7f092c005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:02.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.384+0000 7f0932ffd640 1 -- 172.21.15.26:0/1528130518 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09380d6020 con 0x7f093c06a840 2024-04-21T10:41:02.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.568+0000 7f0943db2640 1 -- 172.21.15.26:0/1528130518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f093c06b1f0 con 0x7f093c06a840 2024-04-21T10:41:02.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.569+0000 7f0932ffd640 1 -- 172.21.15.26:0/1528130518 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f093809d2d0 con 0x7f093c06a840 2024-04-21T10:41:02.568 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-21T10:41:02.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.571+0000 7f0943db2640 1 -- 172.21.15.26:0/1528130518 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f09280a3da0 msgr2=0x7f09280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.571+0000 7f0943db2640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f09280a3da0 0x7f09280a6260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f093c108710 tx=0x7f092c005f90 comp rx=0 tx=0).stop 2024-04-21T10:41:02.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.571+0000 7f0943db2640 1 -- 172.21.15.26:0/1528130518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f093c06a840 msgr2=0x7f093c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.571+0000 7f0943db2640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f093c06a840 0x7f093c1986f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f093800e9f0 tx=0x7f093800eeb0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.571+0000 7f0943db2640 1 -- 172.21.15.26:0/1528130518 shutdown_connections 2024-04-21T10:41:02.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.571+0000 7f0943db2640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f09280a3da0 0x7f09280a6260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.571+0000 7f0943db2640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f093c19cf60 0x7f093c19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.571+0000 7f0943db2640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f093c107270 0x7f093c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.571+0000 7f0943db2640 1 --2- 172.21.15.26:0/1528130518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f093c06a840 0x7f093c1986f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.571+0000 7f0943db2640 1 -- 172.21.15.26:0/1528130518 >> 172.21.15.26:0/1528130518 conn(0x7f093c077eb0 msgr2=0x7f093c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:02.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.571+0000 7f0943db2640 1 -- 172.21.15.26:0/1528130518 shutdown_connections 2024-04-21T10:41:02.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.571+0000 7f0943db2640 1 -- 172.21.15.26:0/1528130518 wait complete. 2024-04-21T10:41:02.580 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:02 smithi026 ceph-mon[27965]: from='client.15834 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:41:02.580 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:02 smithi026 ceph-mon[27965]: from='client.15840 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:41:02.580 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:02 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/4192840952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:41:02.580 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:02 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1916762274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:41:02.580 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:02 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/886070095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:41:02.581 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-21T10:41:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.684+0000 7f9485523640 1 -- 172.21.15.26:0/1683808177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94801008a0 msgr2=0x7f9480100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.684+0000 7f9485523640 1 --2- 172.21.15.26:0/1683808177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94801008a0 0x7f9480100d20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9474009900 tx=0x7f947402bcd0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.684+0000 7f9485523640 1 -- 172.21.15.26:0/1683808177 shutdown_connections 2024-04-21T10:41:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.684+0000 7f9485523640 1 --2- 172.21.15.26:0/1683808177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9480101260 0x7f948010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.684+0000 7f9485523640 1 --2- 172.21.15.26:0/1683808177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94801008a0 0x7f9480100d20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.684+0000 7f9485523640 1 --2- 172.21.15.26:0/1683808177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9480106710 0x7f9480106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.684+0000 7f9485523640 1 -- 172.21.15.26:0/1683808177 >> 172.21.15.26:0/1683808177 conn(0x7f9480076130 msgr2=0x7f9480076540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:02.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.684+0000 7f9485523640 1 -- 172.21.15.26:0/1683808177 shutdown_connections 2024-04-21T10:41:02.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.684+0000 7f9485523640 1 -- 172.21.15.26:0/1683808177 wait complete. 2024-04-21T10:41:02.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.684+0000 7f9485523640 1 Processor -- start 2024-04-21T10:41:02.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f9485523640 1 -- start start 2024-04-21T10:41:02.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f9485523640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94801008a0 0x7f9480198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:02.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f9485523640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9480101260 0x7f9480198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:02.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f9485523640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9480106710 0x7f948019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:02.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f9485523640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f948010ffe0 con 0x7f9480101260 2024-04-21T10:41:02.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f9485523640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f948010fe60 con 0x7f94801008a0 2024-04-21T10:41:02.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f9485523640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9480110160 con 0x7f9480106710 2024-04-21T10:41:02.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f947e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9480101260 0x7f9480198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:02.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f947e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9480101260 0x7f9480198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34824/0 (socket says 172.21.15.26:34824) 2024-04-21T10:41:02.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f947e7fc640 1 -- 172.21.15.26:0/1161065628 learned_addr learned my addr 172.21.15.26:0/1161065628 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:02.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f947effd640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94801008a0 0x7f9480198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:02.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f947f7fe640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9480106710 0x7f948019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:02.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f947e7fc640 1 -- 172.21.15.26:0/1161065628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9480106710 msgr2=0x7f948019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f947e7fc640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9480106710 0x7f948019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f947e7fc640 1 -- 172.21.15.26:0/1161065628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94801008a0 msgr2=0x7f9480198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:02.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f947e7fc640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94801008a0 0x7f9480198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:02.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f947e7fc640 1 -- 172.21.15.26:0/1161065628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948019d720 con 0x7f9480101260 2024-04-21T10:41:02.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f947f7fe640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9480106710 0x7f948019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:41:02.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.685+0000 7f947e7fc640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9480101260 0x7f9480198c70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9474005940 tx=0x7f947402d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:02.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.686+0000 7f9467fff640 1 -- 172.21.15.26:0/1161065628 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474043020 con 0x7f9480101260 2024-04-21T10:41:02.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.686+0000 7f9485523640 1 -- 172.21.15.26:0/1161065628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948019d9b0 con 0x7f9480101260 2024-04-21T10:41:02.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.686+0000 7f9467fff640 1 -- 172.21.15.26:0/1161065628 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f947402dc90 con 0x7f9480101260 2024-04-21T10:41:02.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.686+0000 7f9467fff640 1 -- 172.21.15.26:0/1161065628 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474003ab0 con 0x7f9480101260 2024-04-21T10:41:02.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.686+0000 7f9485523640 1 -- 172.21.15.26:0/1161065628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94801a52a0 con 0x7f9480101260 2024-04-21T10:41:02.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.687+0000 7f9485523640 1 -- 172.21.15.26:0/1161065628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94800691c0 con 0x7f9480101260 2024-04-21T10:41:02.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.688+0000 7f9467fff640 1 -- 172.21.15.26:0/1161065628 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9474046020 con 0x7f9480101260 2024-04-21T10:41:02.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.688+0000 7f9467fff640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f94680a3d20 0x7f94680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:02.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.688+0000 7f947effd640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f94680a3d20 0x7f94680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:02.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.688+0000 7f9467fff640 1 -- 172.21.15.26:0/1161065628 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8550+0+0 (secure 0 0 0) 0x7f94740f2900 con 0x7f9480101260 2024-04-21T10:41:02.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.689+0000 7f947effd640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f94680a3d20 0x7f94680a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9470005fd0 tx=0x7f947000d5b0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:02.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.693+0000 7f9467fff640 1 -- 172.21.15.26:0/1161065628 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f947404d020 con 0x7f9480101260 2024-04-21T10:41:02.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:02 smithi169 ceph-mon[31469]: from='client.15834 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:41:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:02 smithi169 ceph-mon[31469]: from='client.15840 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:41:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:02 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/4192840952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:41:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:02 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1916762274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:41:02.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:02 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/886070095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:41:02.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:02 smithi190 ceph-mon[31423]: from='client.15834 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:41:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:02 smithi190 ceph-mon[31423]: from='client.15840 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T10:41:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:02 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/4192840952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:41:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:02 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1916762274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:41:02.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:02 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/886070095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:41:02.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.930+0000 7f9485523640 1 -- 172.21.15.26:0/1161065628 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f9480108ac0 con 0x7f9480101260 2024-04-21T10:41:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.939+0000 7f9467fff640 1 -- 172.21.15.26:0/1161065628 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+746060 (secure 0 0 0) 0x7f94740bf090 con 0x7f9480101260 2024-04-21T10:41:02.950 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:41:02.950 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:41:02.950 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 25, 2024-04-21T10:41:02.950 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:02.951 INFO:teuthology.orchestra.run.smithi026.stdout: "active_gid": 24869, 2024-04-21T10:41:02.951 INFO:teuthology.orchestra.run.smithi026.stdout: "active_name": "x", 2024-04-21T10:41:02.951 INFO:teuthology.orchestra.run.smithi026.stdout: "active_addrs": { 2024-04-21T10:41:02.951 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-04-21T10:41:02.951 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:02.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-04-21T10:41:02.951 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6800", 2024-04-21T10:41:02.952 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 4245046040 2024-04-21T10:41:02.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:02.952 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-04-21T10:41:02.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6801", 2024-04-21T10:41:02.952 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 4245046040 2024-04-21T10:41:02.952 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:02.953 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-21T10:41:02.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.953 INFO:teuthology.orchestra.run.smithi026.stdout: "active_addr": "172.21.15.26:6801/4245046040", 2024-04-21T10:41:02.953 INFO:teuthology.orchestra.run.smithi026.stdout: "active_change": "2024-04-21T10:30:57.801263+0000", 2024-04-21T10:41:02.953 INFO:teuthology.orchestra.run.smithi026.stdout: "active_mgr_features": 4540701547738038271, 2024-04-21T10:41:02.953 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-04-21T10:41:02.954 INFO:teuthology.orchestra.run.smithi026.stdout: "standbys": [ 2024-04-21T10:41:02.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:02.954 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 15279, 2024-04-21T10:41:02.954 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "y", 2024-04-21T10:41:02.954 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr_features": 4540701547738038271, 2024-04-21T10:41:02.954 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2024-04-21T10:41:02.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:02.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2024-04-21T10:41:02.955 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:02.955 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:02.955 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:02.955 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:02.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:02.955 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:02.955 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T10:41:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.956 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.957 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.957 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:02.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:02.957 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.957 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.957 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.957 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:02.958 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.958 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.958 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:02.958 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:02.958 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:02.958 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:02.958 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:02.958 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:02.959 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:02.959 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:02.959 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:02.959 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.959 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.959 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.959 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:02.960 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:02.960 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:02.960 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.960 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.960 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:02.960 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.960 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.960 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:02.961 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:02.962 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:02.963 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.964 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.964 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.964 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:02.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:02.964 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:02.964 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.964 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.964 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:02.965 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.965 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.965 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.965 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:02.965 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.965 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.965 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.966 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2024-04-21T10:41:02.966 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2024-04-21T10:41:02.966 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.966 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.966 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.966 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:02.966 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.966 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.967 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.967 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2024-04-21T10:41:02.967 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.967 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.967 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.967 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.967 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2024-04-21T10:41:02.967 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2024-04-21T10:41:02.967 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.968 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.968 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.968 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2024-04-21T10:41:02.968 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.968 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.968 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.968 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2024-04-21T10:41:02.968 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.968 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.969 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.969 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.969 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2024-04-21T10:41:02.969 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2024-04-21T10:41:02.969 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.969 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.969 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.969 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:02.970 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.970 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.970 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.970 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2024-04-21T10:41:02.970 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.970 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.970 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.970 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2024-04-21T10:41:02.971 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2024-04-21T10:41:02.971 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.971 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.971 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.971 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:02.971 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.971 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.971 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.971 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2024-04-21T10:41:02.972 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.972 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.972 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.972 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2024-04-21T10:41:02.972 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2024-04-21T10:41:02.972 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:02.972 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.972 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.972 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2024-04-21T10:41:02.973 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.973 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.973 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.973 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2024-04-21T10:41:02.973 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.973 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.973 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.973 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2024-04-21T10:41:02.974 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2024-04-21T10:41:02.974 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.974 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.974 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.974 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:02.974 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.974 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.975 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.975 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2024-04-21T10:41:02.975 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.975 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.975 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.975 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2024-04-21T10:41:02.975 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2024-04-21T10:41:02.976 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:02.976 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.976 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.976 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:02.976 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.976 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.976 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.976 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T10:41:02.977 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.977 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.977 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.977 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2024-04-21T10:41:02.977 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2024-04-21T10:41:02.977 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.977 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.977 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.978 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:02.978 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.978 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.978 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.978 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2024-04-21T10:41:02.978 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.978 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.979 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.979 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:02.979 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:02.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.979 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:02.979 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2024-04-21T10:41:02.979 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:02.979 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:02.979 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:02.980 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2024-04-21T10:41:02.980 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2024-04-21T10:41:02.980 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:02.980 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.980 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.980 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:02.980 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.980 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.980 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.981 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T10:41:02.981 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.981 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.981 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.981 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2024-04-21T10:41:02.981 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2024-04-21T10:41:02.981 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.981 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.982 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.982 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2024-04-21T10:41:02.982 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.982 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.982 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.982 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T10:41:02.982 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T10:41:02.982 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.982 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.983 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2024-04-21T10:41:02.983 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2024-04-21T10:41:02.983 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:02.983 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.983 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.983 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:02.983 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-04-21T10:41:02.984 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-04-21T10:41:02.984 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.984 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T10:41:02.984 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T10:41:02.984 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.984 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.985 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2024-04-21T10:41:02.985 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2024-04-21T10:41:02.985 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:02.985 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.985 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.985 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2024-04-21T10:41:02.985 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-04-21T10:41:02.985 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2024-04-21T10:41:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T10:41:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2024-04-21T10:41:02.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2024-04-21T10:41:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T10:41:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T10:41:02.987 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T10:41:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2024-04-21T10:41:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2024-04-21T10:41:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.988 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2024-04-21T10:41:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2024-04-21T10:41:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2024-04-21T10:41:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2024-04-21T10:41:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T10:41:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.989 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2024-04-21T10:41:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2024-04-21T10:41:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2024-04-21T10:41:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.990 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T10:41:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T10:41:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2024-04-21T10:41:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2024-04-21T10:41:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:02.991 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-04-21T10:41:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-04-21T10:41:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T10:41:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T10:41:02.992 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.993 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:02.994 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.995 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.996 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:02.997 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:02.998 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:02.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:02.998 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:02.998 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:02.998 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:02.998 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:02.998 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:02.998 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:02.999 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:02.999 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:02.999 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:02.999 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:02.999 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:02.999 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:02.999 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:02.999 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.000 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.000 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.000 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.000 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.000 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.000 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.000 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.001 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.001 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.001 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.001 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.001 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.001 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.001 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.001 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.002 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.002 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.002 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.002 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.002 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2024-04-21T10:41:03.003 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2024-04-21T10:41:03.003 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:03.003 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.003 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.003 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:03.003 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.004 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.004 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.004 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T10:41:03.004 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.004 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.004 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.005 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-04-21T10:41:03.005 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-04-21T10:41:03.005 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.005 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.005 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.005 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2024-04-21T10:41:03.005 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.006 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.006 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.006 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2024-04-21T10:41:03.006 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2024-04-21T10:41:03.006 INFO:teuthology.orchestra.run.smithi026.stdout: "read", 2024-04-21T10:41:03.006 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap", 2024-04-21T10:41:03.006 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap-read" 2024-04-21T10:41:03.007 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.007 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2024-04-21T10:41:03.007 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.007 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.007 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.008 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2024-04-21T10:41:03.008 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2024-04-21T10:41:03.008 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.008 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.008 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.008 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.008 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.009 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.009 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.009 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T10:41:03.009 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.009 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.009 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.009 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:03.010 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:03.010 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.010 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.010 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.010 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:03.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.010 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.010 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.011 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T10:41:03.011 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.011 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.011 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.011 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2024-04-21T10:41:03.011 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2024-04-21T10:41:03.012 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.012 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.012 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.012 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-04-21T10:41:03.012 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-04-21T10:41:03.012 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.012 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.012 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T10:41:03.013 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T10:41:03.013 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.013 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.013 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2024-04-21T10:41:03.013 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2024-04-21T10:41:03.013 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:03.013 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.014 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.014 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.014 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.014 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.014 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.014 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T10:41:03.014 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.015 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.015 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.015 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.015 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.015 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2024-04-21T10:41:03.015 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.016 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.016 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.016 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2024-04-21T10:41:03.016 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2024-04-21T10:41:03.016 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:03.016 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.016 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.017 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-04-21T10:41:03.017 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.017 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.017 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.017 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T10:41:03.017 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.017 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.018 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.018 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2024-04-21T10:41:03.018 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2024-04-21T10:41:03.018 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.018 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.018 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.019 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2024-04-21T10:41:03.019 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.019 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.019 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.019 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T10:41:03.019 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.019 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.019 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.020 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2024-04-21T10:41:03.020 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2024-04-21T10:41:03.020 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.020 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.020 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.020 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2024-04-21T10:41:03.021 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.021 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.021 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.021 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T10:41:03.021 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.021 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.021 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.022 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2024-04-21T10:41:03.022 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2024-04-21T10:41:03.022 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.022 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.022 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.022 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.023 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.023 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.023 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.023 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T10:41:03.023 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T10:41:03.023 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.023 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.024 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2024-04-21T10:41:03.024 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2024-04-21T10:41:03.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.024 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.024 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.024 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:03.024 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.024 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.025 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.025 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T10:41:03.025 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.025 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.025 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.025 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2024-04-21T10:41:03.026 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T10:41:03.026 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:03.026 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.026 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.026 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2024-04-21T10:41:03.026 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.026 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.027 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.027 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T10:41:03.027 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.027 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.027 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.027 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2024-04-21T10:41:03.028 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2024-04-21T10:41:03.028 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.028 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.028 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.028 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.028 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.028 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.028 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.029 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2024-04-21T10:41:03.029 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2024-04-21T10:41:03.029 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2024-04-21T10:41:03.029 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.029 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T10:41:03.029 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.029 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.029 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.030 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2024-04-21T10:41:03.030 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2024-04-21T10:41:03.030 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.030 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.030 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.031 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.031 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.031 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.031 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.031 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T10:41:03.031 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.031 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.031 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.032 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2024-04-21T10:41:03.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2024-04-21T10:41:03.032 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.032 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.032 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.032 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.032 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.032 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.033 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.033 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T10:41:03.033 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.033 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.033 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.033 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2024-04-21T10:41:03.033 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2024-04-21T10:41:03.034 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.034 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.034 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.034 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.034 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.034 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.034 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.035 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T10:41:03.035 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.035 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.035 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.035 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2024-04-21T10:41:03.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2024-04-21T10:41:03.035 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.036 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.036 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.036 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T10:41:03.036 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.036 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.036 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.036 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-04-21T10:41:03.036 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.037 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.037 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.037 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2024-04-21T10:41:03.037 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2024-04-21T10:41:03.037 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.037 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.037 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.038 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T10:41:03.038 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.038 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.038 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.038 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2024-04-21T10:41:03.038 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.038 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.038 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.039 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2024-04-21T10:41:03.039 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2024-04-21T10:41:03.039 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.039 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.039 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.039 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T10:41:03.039 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.039 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2024-04-21T10:41:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2024-04-21T10:41:03.040 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2024-04-21T10:41:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T10:41:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-04-21T10:41:03.041 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2024-04-21T10:41:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2024-04-21T10:41:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.042 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.043 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T10:41:03.043 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.043 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.043 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.043 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2024-04-21T10:41:03.043 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.043 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.043 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.044 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2024-04-21T10:41:03.044 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2024-04-21T10:41:03.044 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.044 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.044 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.044 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T10:41:03.044 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.044 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.045 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.045 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2024-04-21T10:41:03.045 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.045 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.045 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.045 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2024-04-21T10:41:03.045 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2024-04-21T10:41:03.046 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.046 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.046 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.046 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T10:41:03.046 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.046 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.046 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.046 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2024-04-21T10:41:03.047 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.047 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.047 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.047 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2024-04-21T10:41:03.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2024-04-21T10:41:03.047 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.047 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.048 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.048 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T10:41:03.048 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.048 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.048 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.048 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2024-04-21T10:41:03.048 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.048 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.048 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.049 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2024-04-21T10:41:03.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2024-04-21T10:41:03.049 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.049 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.049 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.049 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T10:41:03.049 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.050 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.050 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.050 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2024-04-21T10:41:03.050 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.050 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.050 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.051 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2024-04-21T10:41:03.051 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2024-04-21T10:41:03.051 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.051 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.051 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.051 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T10:41:03.051 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.052 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.052 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.052 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2024-04-21T10:41:03.052 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.052 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.052 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.053 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2024-04-21T10:41:03.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2024-04-21T10:41:03.053 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.053 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.053 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.053 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T10:41:03.053 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.053 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.054 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.054 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-04-21T10:41:03.054 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.054 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.054 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.055 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2024-04-21T10:41:03.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2024-04-21T10:41:03.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.055 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.055 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.055 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T10:41:03.055 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.056 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.056 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.056 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2024-04-21T10:41:03.056 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.056 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.056 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.057 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2024-04-21T10:41:03.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2024-04-21T10:41:03.057 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.057 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.057 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.057 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T10:41:03.058 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.058 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.058 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.058 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-04-21T10:41:03.058 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.058 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.058 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.059 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2024-04-21T10:41:03.059 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2024-04-21T10:41:03.059 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.059 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.059 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.059 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T10:41:03.060 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.060 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.060 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.060 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2024-04-21T10:41:03.060 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.060 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.060 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.061 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_samba": { 2024-04-21T10:41:03.061 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_samba", 2024-04-21T10:41:03.061 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.061 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.061 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.061 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T10:41:03.061 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.062 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.062 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.062 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Samba/SMB container image", 2024-04-21T10:41:03.062 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.062 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.062 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.063 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2024-04-21T10:41:03.063 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2024-04-21T10:41:03.063 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.063 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.063 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.063 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T10:41:03.063 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.063 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.064 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.064 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2024-04-21T10:41:03.064 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.064 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.064 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.064 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2024-04-21T10:41:03.065 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2024-04-21T10:41:03.065 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.065 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.065 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.065 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.065 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.066 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.066 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.066 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T10:41:03.066 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.066 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.066 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.066 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2024-04-21T10:41:03.067 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2024-04-21T10:41:03.067 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.067 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.067 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.067 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:03.067 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.067 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.068 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.068 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T10:41:03.068 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.068 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.068 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.068 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2024-04-21T10:41:03.069 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T10:41:03.069 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.069 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.069 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.069 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2024-04-21T10:41:03.069 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.070 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.070 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.070 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T10:41:03.070 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.070 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.070 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.071 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2024-04-21T10:41:03.071 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2024-04-21T10:41:03.071 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.071 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.071 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.071 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2024-04-21T10:41:03.072 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.072 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.072 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.072 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T10:41:03.072 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.072 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.072 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.073 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2024-04-21T10:41:03.073 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2024-04-21T10:41:03.073 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.073 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.073 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.073 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2024-04-21T10:41:03.073 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.074 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.074 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.074 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2024-04-21T10:41:03.074 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.074 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.074 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.075 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2024-04-21T10:41:03.075 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2024-04-21T10:41:03.075 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.075 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.075 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.075 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.075 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.076 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.076 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.076 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T10:41:03.076 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.076 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.076 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.077 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2024-04-21T10:41:03.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2024-04-21T10:41:03.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.077 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.077 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.077 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:03.078 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.078 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.078 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.078 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2024-04-21T10:41:03.078 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.078 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.079 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.079 INFO:teuthology.orchestra.run.smithi026.stdout: "grafana_dashboards_path": { 2024-04-21T10:41:03.079 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "grafana_dashboards_path", 2024-04-21T10:41:03.079 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.079 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.079 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.080 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T10:41:03.080 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.080 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.080 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.080 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T10:41:03.080 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.080 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.081 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.081 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2024-04-21T10:41:03.081 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2024-04-21T10:41:03.081 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.081 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.082 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.082 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:03.082 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.082 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.082 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.082 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2024-04-21T10:41:03.083 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.083 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.083 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.083 INFO:teuthology.orchestra.run.smithi026.stdout: "hw_monitoring": { 2024-04-21T10:41:03.083 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hw_monitoring", 2024-04-21T10:41:03.083 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.084 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.084 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.084 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.084 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.084 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.084 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.084 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T10:41:03.085 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.085 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.085 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.085 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2024-04-21T10:41:03.085 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2024-04-21T10:41:03.085 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.086 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.086 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.086 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.086 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.086 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.086 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.086 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T10:41:03.087 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.087 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.087 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.087 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.087 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.087 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.088 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.088 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.088 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.088 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.088 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.088 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.088 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.088 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.089 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.089 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.089 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.089 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.089 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.089 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.089 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.089 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.090 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.090 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2024-04-21T10:41:03.090 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2024-04-21T10:41:03.090 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.090 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.090 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.091 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.091 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.091 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.091 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.091 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T10:41:03.091 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.092 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.092 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.092 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.092 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.092 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.092 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.093 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.093 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.093 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.093 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.093 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.093 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T10:41:03.093 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.093 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.094 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.094 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.094 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.094 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.094 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.094 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.095 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.095 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.095 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.095 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.095 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.095 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.095 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.095 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.096 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.096 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.096 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.096 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.096 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.096 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.096 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.097 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.097 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.097 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.097 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.097 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.097 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.098 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.098 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.098 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.098 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.098 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.098 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.098 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.099 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T10:41:03.099 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T10:41:03.099 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.099 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.099 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.099 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.099 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.100 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.100 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.100 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T10:41:03.100 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.100 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.100 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.100 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T10:41:03.101 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T10:41:03.101 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.101 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.101 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.101 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2024-04-21T10:41:03.101 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.101 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.101 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.102 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T10:41:03.102 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.102 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.102 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.102 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2024-04-21T10:41:03.102 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2024-04-21T10:41:03.102 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.103 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.103 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.103 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.103 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.103 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.103 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.103 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2024-04-21T10:41:03.103 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.104 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.104 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.104 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2024-04-21T10:41:03.104 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2024-04-21T10:41:03.104 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.104 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.104 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.105 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.105 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.105 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.105 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.105 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T10:41:03.105 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.105 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.105 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.106 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2024-04-21T10:41:03.106 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2024-04-21T10:41:03.106 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.106 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.106 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.106 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.106 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.107 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.107 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.107 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2024-04-21T10:41:03.107 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.107 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.107 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.107 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.107 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-04-21T10:41:03.108 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-04-21T10:41:03.108 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.108 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.108 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.108 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2024-04-21T10:41:03.108 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.108 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.108 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.109 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2024-04-21T10:41:03.109 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2024-04-21T10:41:03.109 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.109 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T10:41:03.109 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.109 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.109 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.110 INFO:teuthology.orchestra.run.smithi026.stdout: "oob_default_addr": { 2024-04-21T10:41:03.110 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "oob_default_addr", 2024-04-21T10:41:03.110 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.110 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.110 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.110 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "169.254.1.1", 2024-04-21T10:41:03.110 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.110 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.111 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.111 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T10:41:03.111 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.111 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.111 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.111 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2024-04-21T10:41:03.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2024-04-21T10:41:03.111 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.112 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.112 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.112 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T10:41:03.112 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.112 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.112 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.112 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T10:41:03.112 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.113 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.113 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.113 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2024-04-21T10:41:03.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2024-04-21T10:41:03.113 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.113 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.113 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.114 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.114 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.114 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.114 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.114 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T10:41:03.114 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.114 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.115 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.115 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2024-04-21T10:41:03.115 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2024-04-21T10:41:03.115 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.115 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.115 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.115 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.116 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.116 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.116 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.116 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T10:41:03.116 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.116 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.116 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.116 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2024-04-21T10:41:03.117 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2024-04-21T10:41:03.117 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.117 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.117 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.117 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.117 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.117 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.117 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.118 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T10:41:03.118 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.118 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.118 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.118 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2024-04-21T10:41:03.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2024-04-21T10:41:03.118 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.119 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.119 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.119 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.119 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.119 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.119 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.119 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T10:41:03.119 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.119 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.120 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.120 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2024-04-21T10:41:03.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2024-04-21T10:41:03.120 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.120 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.120 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.120 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.120 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.121 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.121 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.121 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T10:41:03.121 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.121 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.121 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.121 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2024-04-21T10:41:03.121 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2024-04-21T10:41:03.122 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.122 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.122 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.122 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2024-04-21T10:41:03.122 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.122 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.122 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.123 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2024-04-21T10:41:03.123 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.123 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.123 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.123 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2024-04-21T10:41:03.123 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2024-04-21T10:41:03.123 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.124 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.124 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.124 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.124 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.124 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.124 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.125 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T10:41:03.125 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.125 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.125 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.125 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2024-04-21T10:41:03.126 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2024-04-21T10:41:03.126 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.126 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.126 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.126 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.126 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.127 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.127 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.127 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T10:41:03.127 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.127 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.127 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.128 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2024-04-21T10:41:03.128 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2024-04-21T10:41:03.128 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.128 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.128 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.128 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.129 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.129 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.129 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.129 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T10:41:03.129 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.129 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.130 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.130 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2024-04-21T10:41:03.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2024-04-21T10:41:03.130 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.130 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.131 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.131 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.131 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.131 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.131 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.131 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T10:41:03.132 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.132 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.132 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.132 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2024-04-21T10:41:03.132 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2024-04-21T10:41:03.132 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.133 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.133 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.133 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.133 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.133 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.133 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.133 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T10:41:03.133 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.134 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.134 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.134 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2024-04-21T10:41:03.134 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2024-04-21T10:41:03.134 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.135 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.135 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.135 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.135 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.135 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.135 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.135 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T10:41:03.136 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.136 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.136 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.136 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.136 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.136 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.136 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2024-04-21T10:41:03.137 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.137 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.137 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.137 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.137 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.138 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.138 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.138 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.138 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.138 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.138 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.139 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.139 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.139 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.139 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.139 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.141 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.141 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.141 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.142 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.142 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.142 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.142 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.142 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.142 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.143 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.143 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.143 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.143 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.143 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.143 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.144 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.144 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.144 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.144 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.144 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.144 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.145 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.145 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.145 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.145 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.145 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.145 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.145 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.146 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.146 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.146 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.146 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.146 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.146 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.147 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.147 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.147 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.147 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.147 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.148 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.148 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.148 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.148 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.148 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.148 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.148 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.149 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.149 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.149 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.149 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.149 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.149 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.150 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2024-04-21T10:41:03.150 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2024-04-21T10:41:03.150 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.150 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.150 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.150 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2024-04-21T10:41:03.151 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.151 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.151 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.151 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T10:41:03.151 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.151 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.151 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.152 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2024-04-21T10:41:03.152 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2024-04-21T10:41:03.152 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.152 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.152 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.152 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2024-04-21T10:41:03.153 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.153 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.153 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.153 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T10:41:03.153 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.153 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.154 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.154 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2024-04-21T10:41:03.154 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.155 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.155 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.155 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T10:41:03.155 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T10:41:03.155 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.155 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.155 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.156 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.156 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.156 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.156 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.156 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.156 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.157 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.157 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.157 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T10:41:03.157 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T10:41:03.157 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.157 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.158 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.158 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.158 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.158 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.158 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.158 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.158 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.159 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.159 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.159 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T10:41:03.159 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T10:41:03.159 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.159 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.160 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.160 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.160 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.160 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.160 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.160 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.160 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.160 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.161 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.161 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2024-04-21T10:41:03.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T10:41:03.161 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.161 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.161 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.162 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.162 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.162 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.162 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.162 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.162 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.162 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.163 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.163 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T10:41:03.163 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T10:41:03.163 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.163 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.164 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.164 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.164 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.164 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.164 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.164 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.164 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.165 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.165 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.165 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T10:41:03.165 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T10:41:03.165 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.165 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.166 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.166 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.166 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.166 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.166 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.166 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.167 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.167 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.167 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.167 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.167 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T10:41:03.168 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T10:41:03.168 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.168 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.168 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.168 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.168 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.168 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.169 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.169 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.169 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.169 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.169 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.169 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T10:41:03.169 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T10:41:03.170 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.170 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.170 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.170 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.170 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.170 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.170 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.170 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.171 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.171 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.171 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.171 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T10:41:03.171 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T10:41:03.171 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.172 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.172 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.172 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.172 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.172 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.172 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.172 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.172 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.173 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.173 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.173 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T10:41:03.173 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T10:41:03.173 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.173 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.173 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.174 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.174 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.174 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.174 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.174 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.174 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.174 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.175 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.175 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T10:41:03.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T10:41:03.175 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.175 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.175 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.175 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.176 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.176 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.176 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.176 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.176 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.176 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.176 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.177 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T10:41:03.177 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T10:41:03.177 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.177 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.177 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.177 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.177 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.177 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.178 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.178 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.178 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.178 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.178 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.178 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T10:41:03.178 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T10:41:03.179 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.179 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.179 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.179 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.179 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.179 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.179 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.179 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.179 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.180 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.180 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.180 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T10:41:03.180 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T10:41:03.180 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.180 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.180 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.181 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.181 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.181 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.181 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.181 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.181 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.181 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.181 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.181 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.182 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T10:41:03.182 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T10:41:03.182 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.182 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.182 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.182 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-04-21T10:41:03.182 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.182 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.183 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.183 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.183 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.183 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.183 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.183 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T10:41:03.183 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T10:41:03.183 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.184 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.184 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.184 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.184 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.184 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.184 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.184 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.184 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.184 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.185 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.185 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.185 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2024-04-21T10:41:03.185 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2024-04-21T10:41:03.185 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.185 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.185 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.185 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.186 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.186 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.186 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.186 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.186 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.186 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.186 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.186 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T10:41:03.187 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T10:41:03.187 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.187 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.187 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.187 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-04-21T10:41:03.187 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.187 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.187 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T10:41:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T10:41:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.188 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.189 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.189 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.189 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.189 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.189 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.189 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.189 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.189 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.190 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.190 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T10:41:03.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T10:41:03.190 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.190 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.190 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.190 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.191 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.191 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.191 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.191 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.191 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.191 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.191 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.192 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T10:41:03.192 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T10:41:03.192 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.192 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.192 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.192 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.192 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.192 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.192 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.193 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.193 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.193 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.193 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.193 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T10:41:03.193 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T10:41:03.193 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.194 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.194 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.194 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.194 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.194 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.194 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.194 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.194 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.194 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.195 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.195 INFO:teuthology.orchestra.run.smithi026.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T10:41:03.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T10:41:03.195 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.195 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.195 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.195 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "[]", 2024-04-21T10:41:03.195 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.196 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.196 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.196 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.196 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.196 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.196 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.196 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.196 INFO:teuthology.orchestra.run.smithi026.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T10:41:03.197 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T10:41:03.197 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.197 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.197 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.197 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "{}", 2024-04-21T10:41:03.197 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.197 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.197 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.198 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.198 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.198 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.198 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.198 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T10:41:03.198 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T10:41:03.198 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.199 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.199 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.199 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.199 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.199 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.199 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.199 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.199 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.199 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.200 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.200 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T10:41:03.200 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T10:41:03.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.200 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.200 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.200 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.200 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.201 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.201 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.201 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.201 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.201 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.201 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.202 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T10:41:03.202 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T10:41:03.202 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.202 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.202 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.202 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.202 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.202 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.203 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.203 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.203 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.203 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.203 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.203 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T10:41:03.203 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T10:41:03.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.204 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.204 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.204 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.204 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.204 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.204 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.205 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.205 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.205 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.205 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.205 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T10:41:03.205 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T10:41:03.205 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.206 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.206 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.206 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.206 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.206 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.206 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.206 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.207 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.207 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.207 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.207 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T10:41:03.207 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T10:41:03.207 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.208 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.208 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.208 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.208 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.208 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.208 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.208 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.208 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.209 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.209 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.209 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T10:41:03.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T10:41:03.209 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.209 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.210 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.210 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.210 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.210 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.210 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.210 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.210 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.210 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.211 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.211 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.211 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T10:41:03.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T10:41:03.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.211 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.211 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.211 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.212 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.212 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.212 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.212 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.212 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.212 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.212 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.213 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T10:41:03.213 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T10:41:03.213 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.213 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.213 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.213 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.213 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.214 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.214 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.214 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.214 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.214 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.214 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.214 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T10:41:03.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T10:41:03.215 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.215 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.215 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.215 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.215 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.215 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.215 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.216 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.216 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.216 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.216 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.216 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.216 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T10:41:03.216 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T10:41:03.216 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.217 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.217 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.217 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T10:41:03.217 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.217 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.217 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.217 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.217 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.218 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.218 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.218 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T10:41:03.218 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T10:41:03.218 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.218 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.218 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.219 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.219 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.219 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.219 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.219 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.219 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.219 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.220 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.220 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.220 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T10:41:03.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T10:41:03.220 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.220 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.220 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.220 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2024-04-21T10:41:03.221 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.221 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.221 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.221 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.221 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.221 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.221 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.221 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.222 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T10:41:03.222 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T10:41:03.222 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.222 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.222 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.222 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2024-04-21T10:41:03.222 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.222 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.223 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.223 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.223 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.223 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.223 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.223 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.223 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T10:41:03.223 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T10:41:03.224 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.224 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.224 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.224 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.224 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.224 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.224 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.225 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.225 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.225 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.225 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.225 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.225 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T10:41:03.225 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T10:41:03.225 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.226 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.226 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.226 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-04-21T10:41:03.226 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.226 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.226 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.226 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.227 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.227 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.227 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.227 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.227 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T10:41:03.227 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T10:41:03.227 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.227 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.228 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.228 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.228 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.228 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.228 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.228 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.228 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.228 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.229 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.229 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T10:41:03.229 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T10:41:03.229 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.229 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.229 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.229 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.230 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.230 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.230 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.230 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.230 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.230 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.230 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.230 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.231 INFO:teuthology.orchestra.run.smithi026.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T10:41:03.231 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T10:41:03.231 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.231 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.231 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.231 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.231 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.231 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.232 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.232 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.232 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.232 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.232 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.232 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T10:41:03.232 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T10:41:03.233 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.233 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.233 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.233 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:03.233 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.233 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.233 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.233 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.234 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.234 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.234 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.234 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.234 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T10:41:03.234 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T10:41:03.234 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.235 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.235 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.235 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.235 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.235 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.235 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.235 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.235 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.236 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.236 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.236 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T10:41:03.236 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T10:41:03.236 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.236 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.237 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.237 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-04-21T10:41:03.237 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.237 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.237 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.237 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.237 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.237 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.238 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.238 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2024-04-21T10:41:03.238 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2024-04-21T10:41:03.238 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.238 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.238 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.238 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.239 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.239 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.239 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.239 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.239 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.239 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.239 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.240 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2024-04-21T10:41:03.240 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2024-04-21T10:41:03.240 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.240 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.240 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.240 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.240 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.241 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.241 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.241 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.241 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.241 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.241 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.241 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2024-04-21T10:41:03.241 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2024-04-21T10:41:03.242 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.242 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.242 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.242 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.242 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.242 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.242 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.243 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2024-04-21T10:41:03.243 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.243 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.243 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.243 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2024-04-21T10:41:03.243 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2024-04-21T10:41:03.243 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.244 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.244 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.244 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2024-04-21T10:41:03.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.244 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.244 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.244 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.244 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.244 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.245 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.245 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-04-21T10:41:03.245 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-04-21T10:41:03.245 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.245 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.245 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.245 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.246 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.246 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.246 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.246 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.246 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.246 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.246 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.247 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.247 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.247 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.247 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.247 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.247 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.247 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.247 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.248 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.248 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.248 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.248 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.248 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.248 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.248 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.248 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.248 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.249 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.249 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.249 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.249 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.249 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.249 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.249 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.250 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.250 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.250 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.250 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.250 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.250 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.250 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.250 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.250 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.251 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.251 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.251 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.251 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.251 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.251 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.251 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.252 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.252 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.252 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.252 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.252 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.252 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.252 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.252 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.252 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.253 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.253 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.253 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.253 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.253 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.253 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.253 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.253 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.254 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.254 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.254 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.254 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.254 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.254 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.254 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.254 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.255 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.255 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2024-04-21T10:41:03.255 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2024-04-21T10:41:03.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.255 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.255 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.255 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.256 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.256 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.256 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.256 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2024-04-21T10:41:03.256 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.256 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.256 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.256 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2024-04-21T10:41:03.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T10:41:03.257 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.257 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.257 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.257 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.257 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.257 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.257 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.258 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.258 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.258 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.258 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.258 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-04-21T10:41:03.258 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-04-21T10:41:03.258 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.259 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.259 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.259 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-04-21T10:41:03.259 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.259 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.259 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.259 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.259 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.260 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.260 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.260 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-04-21T10:41:03.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-04-21T10:41:03.260 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.260 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.260 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.260 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2024-04-21T10:41:03.261 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.261 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.261 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.261 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.261 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.261 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.261 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.261 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-04-21T10:41:03.262 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-04-21T10:41:03.262 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.262 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.262 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.262 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.262 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.262 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.262 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.263 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.263 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.263 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.263 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.263 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2024-04-21T10:41:03.263 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2024-04-21T10:41:03.263 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.263 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.264 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.264 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2024-04-21T10:41:03.264 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.264 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.264 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.264 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.264 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.264 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.265 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.265 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-04-21T10:41:03.265 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-04-21T10:41:03.265 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.265 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.265 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.265 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2024-04-21T10:41:03.265 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.266 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.266 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.266 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.266 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2024-04-21T10:41:03.266 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.266 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.266 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.266 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.267 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.267 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-04-21T10:41:03.267 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-04-21T10:41:03.267 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.267 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.267 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.267 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-04-21T10:41:03.268 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-04-21T10:41:03.268 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-04-21T10:41:03.268 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.268 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.268 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.268 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.268 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.269 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2024-04-21T10:41:03.269 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2024-04-21T10:41:03.269 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.269 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.269 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.269 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.269 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.269 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.270 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.270 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.270 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.270 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.270 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.270 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.270 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-04-21T10:41:03.271 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.271 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.271 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.271 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2024-04-21T10:41:03.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2024-04-21T10:41:03.272 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.272 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.272 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.272 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.272 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.272 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.272 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.272 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2024-04-21T10:41:03.273 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.273 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.273 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.273 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.273 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.273 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.273 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.274 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.274 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.274 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.274 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.274 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.274 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.274 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.275 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.275 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.275 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.275 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.275 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.275 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.275 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.275 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.276 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.276 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.276 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.276 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.276 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.276 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.276 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.277 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.277 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.277 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.277 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.277 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.277 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.277 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.278 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.278 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.278 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.278 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.278 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.278 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.278 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.278 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.279 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.279 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.279 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.279 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.279 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.279 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.279 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.279 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.280 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.280 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.280 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.280 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.280 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.280 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.281 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.281 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.281 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.281 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.281 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.281 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.281 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.281 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.282 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.282 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.282 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.282 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2024-04-21T10:41:03.282 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2024-04-21T10:41:03.282 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.283 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.283 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.283 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2024-04-21T10:41:03.283 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.283 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.283 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.284 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T10:41:03.284 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.284 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.284 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.284 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2024-04-21T10:41:03.284 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2024-04-21T10:41:03.285 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.285 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.285 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.285 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2024-04-21T10:41:03.285 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.285 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.285 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.285 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T10:41:03.286 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.286 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.286 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.286 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2024-04-21T10:41:03.286 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2024-04-21T10:41:03.286 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.287 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.287 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.287 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2024-04-21T10:41:03.287 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.287 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.287 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.287 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2024-04-21T10:41:03.287 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.288 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.288 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.288 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2024-04-21T10:41:03.288 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2024-04-21T10:41:03.288 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.288 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.289 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.289 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-04-21T10:41:03.289 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.289 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.289 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.289 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T10:41:03.289 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.290 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.290 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.290 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2024-04-21T10:41:03.290 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2024-04-21T10:41:03.290 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.290 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.290 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.291 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.291 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.291 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.291 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.291 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T10:41:03.291 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.291 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.292 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.292 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:03.292 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:03.292 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.292 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.292 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.293 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:03.293 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.293 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.293 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.293 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T10:41:03.293 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.293 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.294 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.294 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2024-04-21T10:41:03.294 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2024-04-21T10:41:03.294 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.294 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.294 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.294 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2024-04-21T10:41:03.295 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.295 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.295 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.295 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T10:41:03.295 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.295 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.295 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.295 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.296 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.296 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2024-04-21T10:41:03.296 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.296 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.296 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.297 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.297 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.297 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.297 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.297 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.297 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.297 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.297 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.298 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.298 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.298 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.298 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.298 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.298 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.298 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.298 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.299 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.299 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.299 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.299 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.299 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.300 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.300 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.300 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.300 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.300 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.300 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.300 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.300 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.300 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.301 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.301 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.301 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.301 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.301 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.301 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.301 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.302 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.302 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.302 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.302 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.302 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.302 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.302 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.303 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.303 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.303 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.303 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.303 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.303 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.303 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.303 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.304 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.304 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.304 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.304 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.304 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.304 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.304 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.304 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2024-04-21T10:41:03.305 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2024-04-21T10:41:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-04-21T10:41:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.306 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.307 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.307 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.307 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.307 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2024-04-21T10:41:03.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2024-04-21T10:41:03.307 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.307 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.307 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2024-04-21T10:41:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.308 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:03.309 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.310 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2024-04-21T10:41:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2024-04-21T10:41:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2024-04-21T10:41:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2024-04-21T10:41:03.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2024-04-21T10:41:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2024-04-21T10:41:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T10:41:03.312 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2024-04-21T10:41:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2024-04-21T10:41:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2024-04-21T10:41:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T10:41:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.314 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2024-04-21T10:41:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2024-04-21T10:41:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.315 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2024-04-21T10:41:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:03.316 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2024-04-21T10:41:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2024-04-21T10:41:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.317 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T10:41:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.318 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.319 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.320 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.321 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.322 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.323 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.324 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.325 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2024-04-21T10:41:03.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2024-04-21T10:41:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.327 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2024-04-21T10:41:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2024-04-21T10:41:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2024-04-21T10:41:03.328 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2024-04-21T10:41:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.329 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2024-04-21T10:41:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-04-21T10:41:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-04-21T10:41:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.330 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2024-04-21T10:41:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T10:41:03.331 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2024-04-21T10:41:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2024-04-21T10:41:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.332 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-04-21T10:41:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-04-21T10:41:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2024-04-21T10:41:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T10:41:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2024-04-21T10:41:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2024-04-21T10:41:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.334 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2024-04-21T10:41:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2024-04-21T10:41:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2024-04-21T10:41:03.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2024-04-21T10:41:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T10:41:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.336 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2024-04-21T10:41:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.337 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.338 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.339 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.340 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.341 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.342 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.343 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.344 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.345 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.346 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.347 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2024-04-21T10:41:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.348 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.349 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.350 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.351 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.352 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.353 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.354 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.355 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.356 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.357 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2024-04-21T10:41:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2024-04-21T10:41:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2024-04-21T10:41:03.359 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2024-04-21T10:41:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.360 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T10:41:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2024-04-21T10:41:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2024-04-21T10:41:03.361 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2024-04-21T10:41:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T10:41:03.362 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.363 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.364 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.366 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.367 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.368 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.369 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.370 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.371 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2024-04-21T10:41:03.372 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2024-04-21T10:41:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2024-04-21T10:41:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.373 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2024-04-21T10:41:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2024-04-21T10:41:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T10:41:03.374 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.375 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.376 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.377 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.378 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.379 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.380 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.381 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.382 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.383 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2024-04-21T10:41:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2024-04-21T10:41:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.384 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2024-04-21T10:41:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T10:41:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.385 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2024-04-21T10:41:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2024-04-21T10:41:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2024-04-21T10:41:03.386 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2024-04-21T10:41:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.387 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2024-04-21T10:41:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2024-04-21T10:41:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2024-04-21T10:41:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.388 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T10:41:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2024-04-21T10:41:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2024-04-21T10:41:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.389 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2024-04-21T10:41:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.390 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2024-04-21T10:41:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2024-04-21T10:41:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2024-04-21T10:41:03.391 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T10:41:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T10:41:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2024-04-21T10:41:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.394 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.395 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.396 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.398 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.399 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.400 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.401 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2024-04-21T10:41:03.402 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.403 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.404 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.405 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.406 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.407 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.408 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.409 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.410 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.411 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2024-04-21T10:41:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.412 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.413 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.414 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.415 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.416 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.417 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.418 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.419 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.419 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.419 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.419 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.419 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.419 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.420 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.420 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.420 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.420 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.420 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.420 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.420 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.421 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.421 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.421 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.421 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.421 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.421 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.421 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.421 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.422 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-04-21T10:41:03.422 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.422 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.422 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.422 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2024-04-21T10:41:03.423 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2024-04-21T10:41:03.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.423 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.423 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.423 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.423 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.423 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.424 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.424 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T10:41:03.424 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.424 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.424 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.424 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.424 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.425 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.425 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.425 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.425 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.425 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.425 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.425 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.426 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.426 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.426 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.426 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.426 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.426 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.426 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.427 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.427 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.427 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.427 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.427 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.427 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.427 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.428 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.428 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.428 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.428 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.428 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.428 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.428 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.428 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.429 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.429 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.429 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.429 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.429 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.429 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.429 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.430 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.430 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.430 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.430 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.430 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.430 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.431 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.431 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.431 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.431 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.431 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.431 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.431 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.431 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.432 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.432 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.432 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.432 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.432 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.432 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.432 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.432 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.433 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.433 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.433 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.433 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.433 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.433 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2024-04-21T10:41:03.433 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-04-21T10:41:03.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.434 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.434 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.434 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.434 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.434 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.434 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.435 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-04-21T10:41:03.435 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2024-04-21T10:41:03.435 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2024-04-21T10:41:03.435 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.435 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2024-04-21T10:41:03.435 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.435 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.435 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.436 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.436 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.436 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.436 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2024-04-21T10:41:03.436 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.436 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.436 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.437 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.437 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.437 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.437 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.437 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.437 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.437 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.438 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.438 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.438 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.438 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.438 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.438 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.438 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.439 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.439 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.439 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.439 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.439 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.439 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.439 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.440 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.440 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.440 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.440 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.440 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.440 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.440 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.440 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.441 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.441 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.441 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.441 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.441 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.441 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.442 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.442 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.442 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.442 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.442 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.442 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.442 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.443 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.443 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.443 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.443 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.443 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.443 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.443 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.443 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.444 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.444 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.444 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.444 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.444 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.444 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.444 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.445 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.445 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.445 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.445 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.445 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.445 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.445 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.445 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.446 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.446 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.446 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.446 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2024-04-21T10:41:03.446 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.446 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.447 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.447 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.447 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.447 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.447 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.447 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.447 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.447 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.448 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.448 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.448 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.448 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.448 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.448 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.448 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.448 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.449 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.449 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.449 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.449 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.449 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.449 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.449 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.449 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.449 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.450 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.450 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.450 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.450 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.450 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.450 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.450 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.450 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.451 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.451 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.451 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.451 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.451 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.451 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.451 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.451 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.452 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.452 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.452 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.452 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.452 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.452 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.452 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.452 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.452 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.453 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.453 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.453 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.453 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.453 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.453 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.453 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.454 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.454 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.454 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.454 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.454 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.454 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.454 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.454 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.455 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.455 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.455 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.455 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.455 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.455 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.456 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2024-04-21T10:41:03.456 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.456 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.456 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.456 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.456 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.456 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.456 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.456 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.457 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.457 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.457 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.457 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.457 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.457 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.457 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.457 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.458 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.458 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.458 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.458 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.458 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.458 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.458 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.458 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.459 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.459 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.459 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.459 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.459 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.459 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.459 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.459 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.459 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.460 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.460 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.460 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.460 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.460 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.460 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.460 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.461 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.461 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.461 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.461 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.461 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.461 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.461 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.461 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.461 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.462 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.462 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.462 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.462 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.462 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.462 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.462 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.463 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.463 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.463 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.463 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.463 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.463 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.463 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.463 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.464 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.464 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.464 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.464 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.464 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.464 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.464 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:03.465 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:03.465 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.465 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.465 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.465 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:03.465 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.465 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.465 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.465 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.466 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.466 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.466 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.466 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.466 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2024-04-21T10:41:03.466 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2024-04-21T10:41:03.466 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:03.466 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.467 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.467 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-04-21T10:41:03.467 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2024-04-21T10:41:03.467 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.467 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.467 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2024-04-21T10:41:03.467 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T10:41:03.467 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.467 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.468 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.468 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.468 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.468 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.468 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2024-04-21T10:41:03.468 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.468 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.468 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.469 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2024-04-21T10:41:03.469 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2024-04-21T10:41:03.469 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.469 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.469 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.469 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.469 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.469 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.470 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.470 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T10:41:03.470 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.470 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.470 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.470 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.470 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-04-21T10:41:03.470 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-04-21T10:41:03.471 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.471 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.471 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.471 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.471 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.471 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.471 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.471 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.472 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.472 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.472 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.472 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.472 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.472 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.473 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.473 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.473 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.473 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.473 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.473 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.473 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.473 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.474 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.474 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.474 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.474 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.474 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.474 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.474 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.474 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.475 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.475 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.475 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.475 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.475 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.475 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.476 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.476 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.476 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.476 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.476 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.476 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.476 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.476 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.477 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.477 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.477 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.477 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.477 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.477 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.478 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.478 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.478 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.478 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.478 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.478 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.478 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.478 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.479 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.479 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.479 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.479 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.479 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.480 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.480 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.480 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.480 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.480 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.480 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.480 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.480 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.481 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.481 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.481 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.481 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.481 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.481 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2024-04-21T10:41:03.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2024-04-21T10:41:03.482 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.482 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.482 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.482 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2024-04-21T10:41:03.482 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.482 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.482 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.482 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2024-04-21T10:41:03.483 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.483 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.483 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.483 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:03.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:03.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.483 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.484 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.484 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-04-21T10:41:03.484 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.484 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.484 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.484 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2024-04-21T10:41:03.484 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.484 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.485 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.485 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.485 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.485 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.485 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2024-04-21T10:41:03.485 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.485 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.486 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.486 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2024-04-21T10:41:03.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2024-04-21T10:41:03.486 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.486 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.486 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.486 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.486 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.487 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.487 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.487 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.487 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.487 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.487 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.487 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2024-04-21T10:41:03.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2024-04-21T10:41:03.488 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.488 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.488 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.488 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.488 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.488 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.488 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.488 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T10:41:03.489 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T10:41:03.489 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.489 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.489 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.489 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.489 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.489 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.490 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.490 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.490 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.490 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.490 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.490 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.490 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.490 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.490 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.491 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.491 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.491 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.491 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.491 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.491 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.491 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.492 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.492 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.492 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.492 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.492 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.492 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.492 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.492 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.493 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.493 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.493 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.493 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.493 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.493 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.493 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.494 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.494 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.494 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.494 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.494 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.494 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.494 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.494 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.495 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.495 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.495 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.495 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.495 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.495 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.495 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.495 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.496 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.496 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.496 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.496 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.496 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.496 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.496 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.497 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.497 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.497 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.497 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.497 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.497 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.498 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.498 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.498 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2024-04-21T10:41:03.498 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2024-04-21T10:41:03.498 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.498 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.498 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.499 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.499 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.499 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.499 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.499 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.499 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.499 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.499 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.500 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T10:41:03.500 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T10:41:03.500 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.500 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.500 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.500 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2024-04-21T10:41:03.500 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.500 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.501 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.501 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.501 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.501 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.501 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.501 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2024-04-21T10:41:03.501 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2024-04-21T10:41:03.502 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:03.502 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.502 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.502 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2024-04-21T10:41:03.502 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.502 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.502 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.503 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.503 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.503 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.503 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.503 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-04-21T10:41:03.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-04-21T10:41:03.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.504 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.504 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.504 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-04-21T10:41:03.504 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.504 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.504 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.504 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T10:41:03.504 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.505 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.505 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.505 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-04-21T10:41:03.505 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-04-21T10:41:03.505 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.505 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.505 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.506 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2024-04-21T10:41:03.506 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.506 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.506 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.506 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T10:41:03.506 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.506 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.506 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.507 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2024-04-21T10:41:03.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2024-04-21T10:41:03.507 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.507 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.507 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.507 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2024-04-21T10:41:03.507 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.508 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.508 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.508 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.508 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.508 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.508 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.508 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-04-21T10:41:03.508 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-04-21T10:41:03.509 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.509 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.509 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.509 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2024-04-21T10:41:03.509 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.509 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.509 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.509 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2024-04-21T10:41:03.510 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2024-04-21T10:41:03.510 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.510 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.510 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.510 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.510 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.510 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-04-21T10:41:03.511 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-04-21T10:41:03.511 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.511 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.511 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.511 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-04-21T10:41:03.511 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-04-21T10:41:03.511 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-04-21T10:41:03.511 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.511 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.512 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.512 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.512 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.512 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.512 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-04-21T10:41:03.513 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.513 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.513 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.513 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.513 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.513 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.513 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.513 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.514 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.514 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.514 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.514 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.514 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.514 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.514 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.515 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.515 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.515 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.515 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.515 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.515 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.515 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.515 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.516 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.516 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.516 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.516 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.516 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.516 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.516 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.517 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.517 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.517 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.517 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.517 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.517 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.517 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.517 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.518 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.518 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.518 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.518 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.518 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.518 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.518 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.519 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.519 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.519 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.519 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.519 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.519 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.519 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.520 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.520 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.520 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.520 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.520 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.520 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.520 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.521 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.521 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.521 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.521 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.521 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.521 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.521 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.521 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.522 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.522 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.522 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.522 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2024-04-21T10:41:03.522 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2024-04-21T10:41:03.522 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.522 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.523 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.523 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.523 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.523 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.523 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.523 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.523 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.523 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.524 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.524 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.524 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2024-04-21T10:41:03.524 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T10:41:03.524 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.524 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.524 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.525 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.525 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.525 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.525 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.525 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.525 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.525 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.525 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.526 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.526 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2024-04-21T10:41:03.526 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2024-04-21T10:41:03.526 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.526 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.527 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.527 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.527 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.527 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.527 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.527 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.527 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.527 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.528 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.528 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.528 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.528 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.528 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.528 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2024-04-21T10:41:03.528 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.529 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.529 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.529 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2024-04-21T10:41:03.529 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2024-04-21T10:41:03.529 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.529 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.529 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.530 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.530 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.530 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.530 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.530 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.530 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.530 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.531 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.531 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-04-21T10:41:03.531 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-04-21T10:41:03.531 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.531 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.531 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.531 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.532 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.532 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.532 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.532 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.532 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.532 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.532 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.533 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.533 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.533 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.533 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.533 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.533 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.533 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.534 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.534 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.534 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.534 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.534 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.534 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.534 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.535 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.535 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.535 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.535 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.535 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.535 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.535 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.535 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.535 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.536 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.536 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.536 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.536 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.536 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.536 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.536 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.536 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.537 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.537 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.537 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.537 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.537 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.537 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.537 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.537 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.538 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.538 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.538 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.538 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.538 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.538 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.538 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.538 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.539 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.539 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.539 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.539 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.539 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.539 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.539 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.540 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.540 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.540 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.540 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.540 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.540 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.541 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.541 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.541 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.541 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.541 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.541 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.541 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-04-21T10:41:03.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-04-21T10:41:03.542 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.542 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.542 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.542 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.542 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.542 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.543 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.543 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.543 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.543 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.543 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.543 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-04-21T10:41:03.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-04-21T10:41:03.544 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.544 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.544 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.544 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.544 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.544 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.544 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.545 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.545 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.545 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.545 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.545 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.545 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.545 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2024-04-21T10:41:03.546 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.546 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.546 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.546 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.546 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.546 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.547 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.547 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.547 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.547 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.547 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.547 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.547 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.547 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.547 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.548 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.548 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.549 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.549 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.550 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.550 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.550 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.550 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.550 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.550 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.551 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.551 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.551 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.551 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.551 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.551 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.551 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.551 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.552 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.552 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.552 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.552 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.552 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.552 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.552 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.553 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.553 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.553 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.553 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.553 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.553 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.553 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.553 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.554 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.554 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.554 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.554 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.554 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.554 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.554 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.555 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.555 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.555 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.555 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.555 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.555 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.555 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.555 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.556 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.556 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.556 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.556 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.556 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.556 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2024-04-21T10:41:03.557 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.557 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.557 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.557 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.557 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.557 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.557 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.557 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.558 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.558 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.558 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.558 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.558 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.558 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.558 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.558 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.559 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.559 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.559 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.559 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.559 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.559 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.559 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.560 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.560 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.560 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.560 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.560 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.560 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.560 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.561 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.561 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.561 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.561 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.561 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.561 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.562 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.562 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.562 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.562 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.562 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.562 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.562 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.562 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.563 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.563 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.563 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.563 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.563 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.563 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.563 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.563 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.564 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.564 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.564 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.564 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.564 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.564 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.565 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.565 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.565 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.565 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.565 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.565 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.565 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2024-04-21T10:41:03.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2024-04-21T10:41:03.566 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.566 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.566 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.566 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2024-04-21T10:41:03.566 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.566 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.567 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.567 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T10:41:03.567 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.567 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.567 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.567 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.567 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.568 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2024-04-21T10:41:03.568 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.568 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.568 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.568 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.568 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.568 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.569 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.569 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.569 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.569 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.569 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.569 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.569 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.569 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.569 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.570 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.570 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.570 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.570 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.570 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.570 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.570 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.570 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.571 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.571 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.571 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.571 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.571 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.571 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.571 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.571 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.572 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.572 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.572 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.572 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.572 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.572 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.572 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.573 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.573 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.573 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.573 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.573 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.573 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.573 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.573 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.574 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.574 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.574 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.574 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.574 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.574 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.574 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.575 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.575 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.575 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.575 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.575 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.575 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.576 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.576 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.576 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.576 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.576 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.576 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.576 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.576 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.577 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.577 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.577 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2024-04-21T10:41:03.577 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2024-04-21T10:41:03.577 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.578 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.578 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.578 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.578 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.578 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.578 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.578 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.578 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.579 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.579 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.579 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2024-04-21T10:41:03.579 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2024-04-21T10:41:03.579 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.579 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.579 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.580 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2024-04-21T10:41:03.580 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.580 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.580 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.580 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.580 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.580 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.580 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.581 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2024-04-21T10:41:03.581 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2024-04-21T10:41:03.581 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.581 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.581 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.581 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.581 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.582 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.582 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.582 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.582 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.582 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.582 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.583 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2024-04-21T10:41:03.583 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2024-04-21T10:41:03.583 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.583 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.583 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.583 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.583 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.583 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.584 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.584 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.584 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.584 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.584 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.584 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2024-04-21T10:41:03.585 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2024-04-21T10:41:03.585 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:03.585 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.585 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.585 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.585 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.585 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.586 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.586 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.586 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.586 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.586 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.586 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2024-04-21T10:41:03.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2024-04-21T10:41:03.587 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.587 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.587 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.587 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.587 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.587 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.587 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.588 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.588 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.588 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.588 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.588 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2024-04-21T10:41:03.588 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2024-04-21T10:41:03.588 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.590 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.590 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.590 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2024-04-21T10:41:03.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2024-04-21T10:41:03.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.590 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.590 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.591 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.591 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.591 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.591 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.591 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.591 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.591 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.592 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.592 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2024-04-21T10:41:03.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2024-04-21T10:41:03.592 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.592 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.592 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.592 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-04-21T10:41:03.593 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2024-04-21T10:41:03.593 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.593 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.593 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.593 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.593 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.594 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2024-04-21T10:41:03.594 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2024-04-21T10:41:03.594 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.594 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.594 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.594 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.595 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.595 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.595 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.595 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.595 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.595 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.595 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2024-04-21T10:41:03.596 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2024-04-21T10:41:03.596 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.596 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.596 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.596 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.596 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.597 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.597 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.597 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.597 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.597 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.597 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.598 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2024-04-21T10:41:03.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2024-04-21T10:41:03.598 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.598 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.598 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.598 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.598 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.599 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.599 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.599 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.599 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.599 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.599 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.599 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.599 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.600 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-04-21T10:41:03.600 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.600 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.600 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.600 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2024-04-21T10:41:03.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2024-04-21T10:41:03.601 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.601 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.601 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.601 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.601 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.601 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.601 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.601 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T10:41:03.602 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.602 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.602 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.602 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2024-04-21T10:41:03.602 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2024-04-21T10:41:03.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.602 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.603 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.603 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.603 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.603 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.603 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.603 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2024-04-21T10:41:03.603 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.604 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.604 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.604 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.604 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.604 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.604 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.605 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.605 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.605 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.605 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.605 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.605 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.605 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.606 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.606 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.606 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.606 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.606 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.606 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.606 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.606 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.607 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.607 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.607 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.607 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.607 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.607 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.607 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.608 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.608 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.608 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.608 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.608 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.608 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.608 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.609 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.609 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.609 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.609 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.609 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.609 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.609 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.609 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.610 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.610 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.610 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.610 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.610 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.610 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.610 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.610 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.610 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.611 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.611 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.611 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.611 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.611 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.611 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.611 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.612 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.612 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.612 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.612 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.612 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.612 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.612 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.612 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.612 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.613 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2024-04-21T10:41:03.613 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.613 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.613 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.613 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.613 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.613 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.614 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.614 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.614 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.614 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.614 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.614 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.614 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.614 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.615 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.615 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.615 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.615 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.615 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.615 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.615 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.615 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.615 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.616 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.616 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.616 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.616 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.616 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.616 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.616 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.617 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.617 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.617 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.617 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.617 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.617 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.617 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.617 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.618 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.618 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.618 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.618 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.618 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.618 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.618 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.618 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.619 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.619 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.619 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.619 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.619 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.619 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.619 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.619 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.620 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.620 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.620 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.620 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.620 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.620 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.620 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.621 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.621 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.621 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.621 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.621 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.621 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.621 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.621 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.622 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.622 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.622 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2024-04-21T10:41:03.622 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.622 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.622 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.622 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.623 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.623 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.623 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.623 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.623 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.623 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.623 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.623 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.624 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.624 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.624 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.624 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.624 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.624 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.624 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.624 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.625 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.625 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.625 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.625 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.625 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.625 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.625 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.626 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.626 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.626 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.626 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.626 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.626 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.627 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.627 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.627 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.627 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.627 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.627 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.628 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.628 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.628 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.628 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.628 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.628 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.628 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.629 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.629 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.629 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.629 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.629 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.629 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.629 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.630 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.630 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.630 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.630 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.630 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.631 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.631 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.631 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.631 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.631 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.631 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.631 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.631 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.632 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.632 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.632 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.632 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.632 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2024-04-21T10:41:03.632 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.633 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.633 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.633 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2024-04-21T10:41:03.633 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2024-04-21T10:41:03.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.633 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.633 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.633 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T10:41:03.634 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.634 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.634 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.634 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.634 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.634 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.634 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.635 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:03.635 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:03.635 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.635 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.635 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.635 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2024-04-21T10:41:03.635 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.636 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.636 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.636 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.636 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.636 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.636 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.636 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.637 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.637 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.637 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.637 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.637 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.637 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.637 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.638 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.638 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.638 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.638 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.638 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.638 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.638 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.638 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.639 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.639 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.639 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.639 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.639 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.639 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.639 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.640 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.640 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.640 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.640 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.640 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.640 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.640 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.640 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.641 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.641 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.641 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.641 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.641 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.641 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.641 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.642 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.642 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.642 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.642 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.642 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.642 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.642 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.643 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.643 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.643 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.643 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.643 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.643 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.643 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.643 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.644 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.644 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.644 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.644 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.644 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.644 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.644 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.644 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.645 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.645 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.645 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.645 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.645 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2024-04-21T10:41:03.646 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.646 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.646 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.646 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2024-04-21T10:41:03.646 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2024-04-21T10:41:03.646 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.646 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.646 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.647 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.647 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.647 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.647 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.647 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T10:41:03.647 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.647 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.647 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.648 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.648 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2024-04-21T10:41:03.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2024-04-21T10:41:03.648 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.648 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.648 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.648 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.648 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.649 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.649 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.649 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T10:41:03.649 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.649 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.649 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.649 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2024-04-21T10:41:03.650 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2024-04-21T10:41:03.650 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.650 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.650 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.650 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.650 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.650 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.651 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.651 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T10:41:03.651 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.651 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.651 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.651 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.651 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2024-04-21T10:41:03.651 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2024-04-21T10:41:03.652 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.652 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.652 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.652 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.652 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.652 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.653 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.653 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T10:41:03.653 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.653 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.653 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.653 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2024-04-21T10:41:03.653 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2024-04-21T10:41:03.654 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.654 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.654 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.654 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.654 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.654 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.654 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.655 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T10:41:03.655 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.655 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.655 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.655 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2024-04-21T10:41:03.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2024-04-21T10:41:03.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.656 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.656 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.656 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.656 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.656 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.656 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.656 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.656 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.657 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.657 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.657 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2024-04-21T10:41:03.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2024-04-21T10:41:03.657 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.657 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.657 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.658 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.658 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.658 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.658 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.658 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.658 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.658 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.658 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.659 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2024-04-21T10:41:03.659 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2024-04-21T10:41:03.659 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.659 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.659 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.659 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T10:41:03.660 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.660 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.660 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.660 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.660 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.660 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.660 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-04-21T10:41:03.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-04-21T10:41:03.661 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.661 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.661 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.661 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.661 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.661 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.662 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.662 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.662 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.662 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.662 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.662 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:03.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:03.663 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.663 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.663 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.663 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2024-04-21T10:41:03.663 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2024-04-21T10:41:03.663 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.663 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.663 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.664 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.664 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.664 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.664 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2024-04-21T10:41:03.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2024-04-21T10:41:03.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.664 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.665 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.665 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2024-04-21T10:41:03.665 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.665 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.665 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.665 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.665 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.665 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.666 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.666 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2024-04-21T10:41:03.666 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2024-04-21T10:41:03.666 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.666 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.666 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.666 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.667 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.667 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.667 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.667 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.667 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.667 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.667 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.668 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2024-04-21T10:41:03.668 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2024-04-21T10:41:03.668 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.668 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.668 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.668 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.668 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.668 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.669 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.669 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.669 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.669 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.669 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.669 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.669 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.670 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.670 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.670 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.670 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.670 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.670 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.670 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.671 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.671 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.671 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.671 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.671 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.671 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.671 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.671 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.671 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.672 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.672 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.672 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.672 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.672 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.672 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.673 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.673 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.673 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.673 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.673 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.673 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.673 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.673 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.674 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.674 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.674 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.674 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.674 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.674 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.674 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.675 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.675 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.675 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.675 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.675 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.675 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.675 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.675 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.676 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.676 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.676 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.676 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.676 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.676 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.677 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.677 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.677 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.677 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.677 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.677 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.677 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.678 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.678 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.678 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.678 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.678 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2024-04-21T10:41:03.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2024-04-21T10:41:03.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.679 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.679 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.679 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.679 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.679 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.679 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.679 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.679 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.680 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.680 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.680 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2024-04-21T10:41:03.680 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2024-04-21T10:41:03.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.680 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.680 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.680 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.681 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.681 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.681 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.681 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.681 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.681 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.681 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.682 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2024-04-21T10:41:03.682 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2024-04-21T10:41:03.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.682 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.682 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.682 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T10:41:03.682 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.682 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.683 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.683 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.683 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.683 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.683 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.683 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.683 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2024-04-21T10:41:03.684 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.684 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.684 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.684 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.684 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.685 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.685 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.685 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.685 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.685 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.685 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.685 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.686 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.686 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.686 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.686 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.686 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.686 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.686 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.687 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.687 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.687 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.687 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.687 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.688 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.688 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.688 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.688 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.688 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.688 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.688 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.688 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.689 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.689 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.689 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.689 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.689 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.690 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.690 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.690 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.690 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.690 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.690 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.690 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.690 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.691 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.691 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.691 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.691 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.691 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.691 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.691 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.692 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.692 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.692 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.692 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.692 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.692 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.692 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.693 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.693 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.693 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.693 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.693 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.693 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.693 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.694 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.694 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-04-21T10:41:03.694 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.694 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.694 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.694 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.694 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.694 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.695 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.695 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.695 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.695 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.695 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.695 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.695 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.695 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.696 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.696 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.696 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.696 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.696 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.696 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.696 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.696 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.697 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.697 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.697 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.697 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.697 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.697 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.697 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.697 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.698 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.698 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.698 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.698 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.698 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.698 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.698 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.699 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.699 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.699 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.699 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.699 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.699 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.699 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.699 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.700 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.700 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.700 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.700 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.700 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.700 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.700 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.700 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.701 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.701 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.701 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.701 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.701 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.701 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.701 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.702 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.702 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.702 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.702 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.702 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.702 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.702 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.702 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.703 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2024-04-21T10:41:03.703 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2024-04-21T10:41:03.703 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.703 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.703 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.703 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2024-04-21T10:41:03.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.704 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.704 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.704 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T10:41:03.704 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.704 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.704 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.704 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2024-04-21T10:41:03.705 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2024-04-21T10:41:03.705 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.705 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.705 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.705 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.705 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.705 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.705 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.706 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2024-04-21T10:41:03.706 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.706 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.706 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.706 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2024-04-21T10:41:03.706 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2024-04-21T10:41:03.707 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.707 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.707 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.707 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:03.707 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.707 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.707 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.707 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T10:41:03.707 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.708 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.708 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.708 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_no_wait": { 2024-04-21T10:41:03.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T10:41:03.708 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.708 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.708 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.709 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.709 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.709 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.709 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.709 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T10:41:03.709 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.709 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.709 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.710 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.710 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.710 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.710 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.710 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2024-04-21T10:41:03.710 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.710 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.710 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.710 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2024-04-21T10:41:03.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2024-04-21T10:41:03.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:03.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.711 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.711 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2024-04-21T10:41:03.711 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.711 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.711 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.712 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.712 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.712 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2024-04-21T10:41:03.712 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2024-04-21T10:41:03.713 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.713 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.713 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.713 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.713 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.713 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.713 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.714 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.714 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.714 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.714 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:03.714 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:03.714 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.714 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.715 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.715 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:03.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.715 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.715 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.715 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.715 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.715 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.716 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.716 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.716 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.716 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.716 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.716 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.717 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.717 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.717 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.717 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.717 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.717 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.717 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.717 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.718 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.718 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.718 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.718 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.718 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.718 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.718 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.719 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.719 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.719 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.719 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.719 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.719 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.719 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.719 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.720 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.720 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.720 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.720 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.720 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.720 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.721 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.721 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.721 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.721 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.721 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.721 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.721 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.722 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.722 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.722 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.722 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.722 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.722 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.722 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.722 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.723 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.723 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.723 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.723 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.723 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.723 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.724 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.724 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.724 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.724 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.724 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.724 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.724 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.725 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.725 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2024-04-21T10:41:03.725 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2024-04-21T10:41:03.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.725 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.725 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.725 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.726 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.726 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.726 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.726 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.726 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.726 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.726 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.727 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2024-04-21T10:41:03.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2024-04-21T10:41:03.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.727 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.727 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.727 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2024-04-21T10:41:03.727 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.727 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.727 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.728 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.728 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.728 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.728 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.728 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2024-04-21T10:41:03.728 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2024-04-21T10:41:03.728 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.729 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.729 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.729 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T10:41:03.729 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.729 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.729 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.729 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.729 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.729 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.730 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.730 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-21T10:41:03.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.730 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 24955, 2024-04-21T10:41:03.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "z", 2024-04-21T10:41:03.731 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr_features": 4540701547738038271, 2024-04-21T10:41:03.731 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2024-04-21T10:41:03.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2024-04-21T10:41:03.731 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.731 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.731 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.731 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:03.732 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:03.732 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.732 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.732 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.732 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:03.732 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.732 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.732 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.733 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T10:41:03.733 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.733 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.733 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.733 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.733 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.733 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.733 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.734 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.734 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.734 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.734 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.734 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.734 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.734 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.734 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.735 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.735 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.735 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.735 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.735 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.735 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.735 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.735 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.736 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.736 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.736 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.736 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.736 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.737 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.737 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.737 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.737 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.737 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.737 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.737 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.738 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.738 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.738 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.738 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.738 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.739 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.739 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.739 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.739 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.739 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.739 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.739 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.739 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.740 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.740 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.740 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.740 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.740 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.740 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.741 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.741 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.741 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.741 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.741 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.741 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.741 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.741 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.741 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.742 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.742 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2024-04-21T10:41:03.742 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2024-04-21T10:41:03.742 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.742 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.742 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.742 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.743 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.743 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.743 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.743 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2024-04-21T10:41:03.743 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.743 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.743 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.743 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2024-04-21T10:41:03.744 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2024-04-21T10:41:03.744 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.744 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.744 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.744 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2024-04-21T10:41:03.744 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.744 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.744 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.744 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2024-04-21T10:41:03.745 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.745 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.745 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.745 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2024-04-21T10:41:03.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2024-04-21T10:41:03.745 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.745 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.746 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.746 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.746 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.746 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.746 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.746 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2024-04-21T10:41:03.746 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.746 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.746 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.747 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2024-04-21T10:41:03.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2024-04-21T10:41:03.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.747 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.747 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.747 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.747 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.748 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.748 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.748 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2024-04-21T10:41:03.748 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.748 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.748 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.748 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2024-04-21T10:41:03.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2024-04-21T10:41:03.749 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.749 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.749 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.749 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2024-04-21T10:41:03.749 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.749 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.749 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.749 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2024-04-21T10:41:03.750 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.750 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.750 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.750 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2024-04-21T10:41:03.750 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2024-04-21T10:41:03.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.750 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.751 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.751 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.753 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.753 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.753 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.753 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2024-04-21T10:41:03.754 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.754 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.754 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.754 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2024-04-21T10:41:03.754 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2024-04-21T10:41:03.754 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T10:41:03.755 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2024-04-21T10:41:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2024-04-21T10:41:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.756 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2024-04-21T10:41:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.757 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2024-04-21T10:41:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.758 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2024-04-21T10:41:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2024-04-21T10:41:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.759 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T10:41:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2024-04-21T10:41:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2024-04-21T10:41:03.760 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2024-04-21T10:41:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T10:41:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T10:41:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2024-04-21T10:41:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2024-04-21T10:41:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:03.762 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-04-21T10:41:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-04-21T10:41:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T10:41:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T10:41:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2024-04-21T10:41:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2024-04-21T10:41:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.764 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2024-04-21T10:41:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-04-21T10:41:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2024-04-21T10:41:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T10:41:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.765 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2024-04-21T10:41:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2024-04-21T10:41:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T10:41:03.766 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T10:41:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T10:41:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2024-04-21T10:41:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2024-04-21T10:41:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2024-04-21T10:41:03.768 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2024-04-21T10:41:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2024-04-21T10:41:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2024-04-21T10:41:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T10:41:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2024-04-21T10:41:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2024-04-21T10:41:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2024-04-21T10:41:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.770 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T10:41:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T10:41:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.771 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2024-04-21T10:41:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2024-04-21T10:41:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-04-21T10:41:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-04-21T10:41:03.772 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T10:41:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T10:41:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.773 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.774 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.775 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.777 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.778 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.779 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2024-04-21T10:41:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2024-04-21T10:41:03.782 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T10:41:03.783 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-04-21T10:41:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-04-21T10:41:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.784 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2024-04-21T10:41:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2024-04-21T10:41:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2024-04-21T10:41:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "read", 2024-04-21T10:41:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap", 2024-04-21T10:41:03.785 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap-read" 2024-04-21T10:41:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2024-04-21T10:41:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2024-04-21T10:41:03.786 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2024-04-21T10:41:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.787 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T10:41:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.788 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T10:41:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.789 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2024-04-21T10:41:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2024-04-21T10:41:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.790 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-04-21T10:41:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-04-21T10:41:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T10:41:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T10:41:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.791 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2024-04-21T10:41:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2024-04-21T10:41:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.792 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T10:41:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2024-04-21T10:41:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2024-04-21T10:41:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2024-04-21T10:41:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.794 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-04-21T10:41:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T10:41:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.795 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2024-04-21T10:41:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2024-04-21T10:41:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2024-04-21T10:41:03.796 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T10:41:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.797 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2024-04-21T10:41:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2024-04-21T10:41:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2024-04-21T10:41:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.798 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T10:41:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2024-04-21T10:41:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2024-04-21T10:41:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.799 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T10:41:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T10:41:03.800 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2024-04-21T10:41:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2024-04-21T10:41:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.801 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T10:41:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.802 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2024-04-21T10:41:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T10:41:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2024-04-21T10:41:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.803 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T10:41:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2024-04-21T10:41:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2024-04-21T10:41:03.804 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2024-04-21T10:41:03.805 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2024-04-21T10:41:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2024-04-21T10:41:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T10:41:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.806 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2024-04-21T10:41:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2024-04-21T10:41:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.807 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T10:41:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2024-04-21T10:41:03.808 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2024-04-21T10:41:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.809 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T10:41:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2024-04-21T10:41:03.810 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2024-04-21T10:41:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.811 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T10:41:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2024-04-21T10:41:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2024-04-21T10:41:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.812 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T10:41:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-04-21T10:41:03.813 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2024-04-21T10:41:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2024-04-21T10:41:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.814 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T10:41:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2024-04-21T10:41:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.815 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2024-04-21T10:41:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2024-04-21T10:41:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.816 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T10:41:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2024-04-21T10:41:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.817 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2024-04-21T10:41:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2024-04-21T10:41:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T10:41:03.818 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-04-21T10:41:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.819 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2024-04-21T10:41:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2024-04-21T10:41:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T10:41:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.820 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2024-04-21T10:41:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2024-04-21T10:41:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2024-04-21T10:41:03.821 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T10:41:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2024-04-21T10:41:03.822 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2024-04-21T10:41:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2024-04-21T10:41:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.823 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T10:41:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2024-04-21T10:41:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.824 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2024-04-21T10:41:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2024-04-21T10:41:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T10:41:03.825 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2024-04-21T10:41:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.826 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2024-04-21T10:41:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2024-04-21T10:41:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T10:41:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.827 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2024-04-21T10:41:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2024-04-21T10:41:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2024-04-21T10:41:03.828 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T10:41:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2024-04-21T10:41:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.829 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2024-04-21T10:41:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2024-04-21T10:41:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.830 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T10:41:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-04-21T10:41:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.831 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2024-04-21T10:41:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2024-04-21T10:41:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.832 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T10:41:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2024-04-21T10:41:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2024-04-21T10:41:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2024-04-21T10:41:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T10:41:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.834 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-04-21T10:41:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2024-04-21T10:41:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2024-04-21T10:41:03.835 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T10:41:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2024-04-21T10:41:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.836 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_samba": { 2024-04-21T10:41:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_samba", 2024-04-21T10:41:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.837 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T10:41:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Samba/SMB container image", 2024-04-21T10:41:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.838 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2024-04-21T10:41:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2024-04-21T10:41:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.839 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T10:41:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2024-04-21T10:41:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2024-04-21T10:41:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2024-04-21T10:41:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.841 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T10:41:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.842 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2024-04-21T10:41:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2024-04-21T10:41:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.843 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T10:41:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2024-04-21T10:41:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T10:41:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2024-04-21T10:41:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T10:41:03.845 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2024-04-21T10:41:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2024-04-21T10:41:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.846 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2024-04-21T10:41:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T10:41:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.847 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2024-04-21T10:41:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2024-04-21T10:41:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.848 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2024-04-21T10:41:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2024-04-21T10:41:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2024-04-21T10:41:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2024-04-21T10:41:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.850 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T10:41:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2024-04-21T10:41:03.851 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2024-04-21T10:41:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.852 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2024-04-21T10:41:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: "grafana_dashboards_path": { 2024-04-21T10:41:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "grafana_dashboards_path", 2024-04-21T10:41:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.853 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.854 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.854 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T10:41:03.854 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.854 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.854 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.854 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T10:41:03.854 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.854 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.855 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.855 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2024-04-21T10:41:03.855 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2024-04-21T10:41:03.855 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.855 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.855 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.855 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:03.856 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.856 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.856 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.856 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2024-04-21T10:41:03.856 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.856 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.856 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.856 INFO:teuthology.orchestra.run.smithi026.stdout: "hw_monitoring": { 2024-04-21T10:41:03.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hw_monitoring", 2024-04-21T10:41:03.857 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.857 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.857 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.857 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.857 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.857 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.857 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.858 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T10:41:03.858 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.858 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.858 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.858 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2024-04-21T10:41:03.858 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2024-04-21T10:41:03.858 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.859 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.859 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.859 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.859 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.859 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.859 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.859 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T10:41:03.859 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.860 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.860 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.860 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.860 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.860 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.860 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.860 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.860 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.861 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.861 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.861 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.861 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.861 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.861 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.861 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.861 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.862 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.862 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.862 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.862 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.862 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.862 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.862 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2024-04-21T10:41:03.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2024-04-21T10:41:03.863 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.863 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.863 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.863 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.863 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.863 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.863 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.864 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T10:41:03.864 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.864 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.864 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.864 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.864 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.865 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.865 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.865 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.865 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.865 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.865 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.865 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.865 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T10:41:03.866 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.866 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.866 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.866 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.866 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.866 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.867 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.867 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.867 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.867 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.867 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.867 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.867 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.867 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.868 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.868 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.868 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.868 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.868 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.868 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.868 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.868 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.869 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.869 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.869 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.869 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.869 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.869 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.870 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.870 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.870 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.870 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.870 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.870 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.870 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.871 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.871 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T10:41:03.871 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T10:41:03.871 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.871 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.871 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.871 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.872 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.872 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.872 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.872 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T10:41:03.872 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.872 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.872 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.873 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T10:41:03.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T10:41:03.873 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.873 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.873 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.873 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2024-04-21T10:41:03.873 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.873 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.874 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.874 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T10:41:03.874 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.874 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.874 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.874 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2024-04-21T10:41:03.874 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2024-04-21T10:41:03.875 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.875 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.875 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.875 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.875 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.875 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.875 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.876 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2024-04-21T10:41:03.876 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.876 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.876 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.876 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2024-04-21T10:41:03.876 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2024-04-21T10:41:03.876 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.877 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.877 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.877 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.877 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.877 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.877 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.877 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T10:41:03.877 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.878 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.878 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.878 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2024-04-21T10:41:03.878 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2024-04-21T10:41:03.878 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.878 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.878 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.879 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.879 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.879 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.879 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.879 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2024-04-21T10:41:03.879 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.879 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.879 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.880 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-04-21T10:41:03.880 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-04-21T10:41:03.880 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.880 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.880 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.880 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2024-04-21T10:41:03.880 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.881 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.881 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.881 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2024-04-21T10:41:03.881 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2024-04-21T10:41:03.881 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.881 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T10:41:03.881 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.881 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.882 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.882 INFO:teuthology.orchestra.run.smithi026.stdout: "oob_default_addr": { 2024-04-21T10:41:03.882 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "oob_default_addr", 2024-04-21T10:41:03.882 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.882 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.882 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.882 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "169.254.1.1", 2024-04-21T10:41:03.883 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.883 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.883 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.883 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T10:41:03.883 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.883 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.883 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.883 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2024-04-21T10:41:03.884 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2024-04-21T10:41:03.884 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.884 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.884 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.884 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T10:41:03.884 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.884 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.884 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.885 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T10:41:03.885 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.885 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.885 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.885 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2024-04-21T10:41:03.885 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2024-04-21T10:41:03.885 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.886 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.886 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.886 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.886 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.886 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.886 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.886 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T10:41:03.886 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.887 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.887 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.887 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2024-04-21T10:41:03.887 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2024-04-21T10:41:03.887 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.887 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.887 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.887 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.888 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.888 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.888 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.888 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T10:41:03.888 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.888 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.888 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.889 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2024-04-21T10:41:03.889 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2024-04-21T10:41:03.889 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.889 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.889 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.889 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.889 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.889 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.889 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.890 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T10:41:03.890 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.890 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.890 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.890 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2024-04-21T10:41:03.890 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2024-04-21T10:41:03.890 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.891 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.891 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.891 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.891 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.891 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.891 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T10:41:03.891 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.892 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.892 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.892 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2024-04-21T10:41:03.892 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2024-04-21T10:41:03.892 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.892 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.893 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.893 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.893 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.893 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.893 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.893 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T10:41:03.893 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.893 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.893 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.894 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2024-04-21T10:41:03.894 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2024-04-21T10:41:03.894 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.894 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.894 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.894 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2024-04-21T10:41:03.895 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.895 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.895 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.895 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2024-04-21T10:41:03.895 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.895 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.895 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.896 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2024-04-21T10:41:03.896 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2024-04-21T10:41:03.896 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.896 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.896 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.896 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.896 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.896 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.897 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.897 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T10:41:03.897 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.897 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.897 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.897 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2024-04-21T10:41:03.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2024-04-21T10:41:03.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.898 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.898 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.898 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.898 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.898 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.898 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.898 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T10:41:03.899 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.899 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.899 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.899 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2024-04-21T10:41:03.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2024-04-21T10:41:03.899 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.899 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.900 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.900 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.900 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.900 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.900 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.900 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T10:41:03.900 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.900 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.901 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.901 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2024-04-21T10:41:03.901 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2024-04-21T10:41:03.901 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.901 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.901 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.901 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.902 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.902 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.902 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.902 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T10:41:03.902 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.902 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.902 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.903 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2024-04-21T10:41:03.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2024-04-21T10:41:03.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.903 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.903 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.904 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.904 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.904 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.904 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.904 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T10:41:03.904 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.904 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.905 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.905 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2024-04-21T10:41:03.905 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2024-04-21T10:41:03.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.905 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.905 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.905 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.906 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.906 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.906 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.906 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T10:41:03.906 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.906 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.906 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.907 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.907 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.907 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.907 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2024-04-21T10:41:03.907 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.907 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.908 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.908 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:03.908 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:03.908 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.908 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.908 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.908 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.908 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.909 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.909 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.909 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.909 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.909 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.909 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.909 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.909 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.910 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.910 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.910 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.910 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.910 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.910 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:03.910 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.911 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.912 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.913 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:03.913 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.913 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.913 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:03.913 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:03.913 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:03.913 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:03.913 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:03.914 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:03.914 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:03.914 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:03.914 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.914 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.914 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.914 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.915 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:03.915 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:03.915 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.915 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.915 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.915 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.915 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.915 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.915 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.916 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.916 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.916 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.916 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.916 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2024-04-21T10:41:03.916 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2024-04-21T10:41:03.916 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.917 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.917 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.917 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2024-04-21T10:41:03.917 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.917 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.917 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.917 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T10:41:03.917 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.917 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.918 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.918 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2024-04-21T10:41:03.918 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2024-04-21T10:41:03.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:03.918 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.918 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:03.918 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2024-04-21T10:41:03.919 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.919 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.919 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.919 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T10:41:03.919 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.919 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.919 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.920 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.920 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:03.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.920 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:03.920 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2024-04-21T10:41:03.920 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:03.920 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:03.920 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:03.921 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T10:41:03.921 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T10:41:03.921 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.921 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.921 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.921 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.921 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.922 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.922 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.922 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.922 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.922 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.922 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.923 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T10:41:03.923 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T10:41:03.923 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.923 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.923 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.923 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.923 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.923 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.924 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.924 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.924 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.924 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.924 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.924 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T10:41:03.925 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T10:41:03.925 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.925 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.925 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.925 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.925 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.925 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.925 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.926 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.926 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.926 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.926 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.926 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2024-04-21T10:41:03.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T10:41:03.927 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.927 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.927 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.927 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.927 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.927 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.927 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.928 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.928 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.928 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.928 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.928 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T10:41:03.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T10:41:03.929 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.929 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.929 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.929 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.929 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.929 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.929 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.929 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.930 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.930 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.930 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.930 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T10:41:03.930 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T10:41:03.930 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.930 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.931 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.931 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.931 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.931 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.931 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.931 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.931 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.931 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.932 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.932 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T10:41:03.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T10:41:03.932 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.932 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.932 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.933 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.933 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.933 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.933 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.933 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.933 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.933 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.933 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.934 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T10:41:03.934 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T10:41:03.934 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.934 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.934 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.934 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.934 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.935 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.935 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.935 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.935 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.935 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.935 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.935 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T10:41:03.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T10:41:03.936 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.936 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.936 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.936 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.936 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.936 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.937 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.937 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.937 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.937 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.937 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.937 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T10:41:03.937 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T10:41:03.937 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.938 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.938 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.938 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.938 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.938 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.938 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.938 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.938 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.939 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.939 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.939 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T10:41:03.939 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T10:41:03.939 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.939 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.939 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.940 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.940 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.940 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.940 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.940 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.940 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.940 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.940 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.941 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T10:41:03.941 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T10:41:03.941 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.941 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.941 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.941 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.941 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.941 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.942 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.942 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.942 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.942 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.942 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.942 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T10:41:03.942 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T10:41:03.942 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.943 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.943 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.943 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.943 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.943 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.943 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.943 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.943 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.944 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.944 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.944 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T10:41:03.944 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T10:41:03.944 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.944 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.944 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.944 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.945 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.945 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.945 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.945 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.945 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.945 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.945 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.945 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T10:41:03.946 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T10:41:03.946 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.946 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.946 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.946 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-04-21T10:41:03.946 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.946 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.946 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.947 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.947 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.947 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.947 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.947 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T10:41:03.947 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T10:41:03.947 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.948 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.948 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.948 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.948 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.948 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.948 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.948 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.948 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.948 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.949 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.949 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2024-04-21T10:41:03.949 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2024-04-21T10:41:03.949 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.949 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.949 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.949 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.950 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.950 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.950 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.950 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.950 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.950 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.950 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.951 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T10:41:03.951 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T10:41:03.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.951 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.951 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.951 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-04-21T10:41:03.951 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.951 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.952 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.952 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.952 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.952 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.952 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.952 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T10:41:03.952 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T10:41:03.952 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.953 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.953 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.953 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.953 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.953 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.953 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.953 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.953 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.953 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.954 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.954 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T10:41:03.954 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T10:41:03.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.954 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.954 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.954 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.955 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.955 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.955 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.955 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.955 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.955 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.955 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.957 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T10:41:03.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T10:41:03.957 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.957 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.957 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.957 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.958 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.958 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.958 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.958 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.958 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.958 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.958 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.958 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T10:41:03.959 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T10:41:03.959 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.959 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.959 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.959 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.959 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.959 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.959 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.960 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.960 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.960 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.960 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.960 INFO:teuthology.orchestra.run.smithi026.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T10:41:03.960 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T10:41:03.960 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.960 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.961 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.961 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "[]", 2024-04-21T10:41:03.961 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.961 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.961 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.961 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.961 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.961 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.962 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.962 INFO:teuthology.orchestra.run.smithi026.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T10:41:03.962 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T10:41:03.962 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.962 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.962 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.962 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "{}", 2024-04-21T10:41:03.963 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.963 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.963 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.963 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.963 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.963 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.963 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.964 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T10:41:03.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T10:41:03.964 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.964 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.964 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.964 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.964 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.964 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.965 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.965 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.965 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.965 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.965 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.965 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T10:41:03.965 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T10:41:03.965 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.966 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.966 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.966 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.966 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.966 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.966 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.966 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.966 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.967 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.967 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.967 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.967 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T10:41:03.967 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T10:41:03.967 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.967 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.967 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.968 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.968 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.968 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.968 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.968 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.968 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.968 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.968 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.969 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T10:41:03.969 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T10:41:03.969 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.969 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.969 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.969 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.969 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.969 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.970 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.970 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.970 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.970 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.970 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.970 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T10:41:03.970 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T10:41:03.971 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.971 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.971 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.971 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.971 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.971 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.971 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.971 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.972 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.972 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.972 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.972 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T10:41:03.972 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T10:41:03.972 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.972 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.973 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.973 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.973 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.973 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.973 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.973 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.973 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.974 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.974 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.974 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T10:41:03.974 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T10:41:03.974 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.975 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.975 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.975 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.975 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.975 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.975 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.975 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.975 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.976 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.976 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.976 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T10:41:03.976 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T10:41:03.976 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.976 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.977 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.977 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.977 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.977 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.977 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.977 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.977 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.978 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.978 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.978 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T10:41:03.978 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T10:41:03.978 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.978 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.979 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.979 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.979 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.979 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.979 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.979 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.979 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.979 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.980 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.980 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T10:41:03.980 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T10:41:03.980 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.980 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.980 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.981 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.982 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T10:41:03.982 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T10:41:03.982 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.982 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.982 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.982 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T10:41:03.982 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.983 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.983 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.983 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.983 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.983 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.983 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.983 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T10:41:03.984 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T10:41:03.984 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.984 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.984 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.984 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:03.984 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.984 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.984 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.985 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.985 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.985 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.985 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.985 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T10:41:03.985 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T10:41:03.985 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.986 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.986 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.986 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2024-04-21T10:41:03.986 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.986 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.986 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.986 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.987 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.987 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.987 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.987 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T10:41:03.987 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T10:41:03.987 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.987 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.988 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.988 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2024-04-21T10:41:03.988 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.988 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.988 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.988 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.988 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.988 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.989 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.989 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T10:41:03.989 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T10:41:03.989 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.989 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.989 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.989 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.990 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.990 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.990 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.990 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.990 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.990 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.990 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.991 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T10:41:03.991 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T10:41:03.991 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.991 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.991 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.991 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-04-21T10:41:03.991 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.992 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.992 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.992 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.992 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.992 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.992 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.992 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T10:41:03.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T10:41:03.993 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:03.993 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.993 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.993 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:03.993 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.993 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.993 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.994 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.994 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.994 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.994 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.994 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T10:41:03.994 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T10:41:03.994 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.995 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.995 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.995 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:03.995 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.995 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.995 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.995 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.995 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.995 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.996 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.996 INFO:teuthology.orchestra.run.smithi026.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T10:41:03.996 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T10:41:03.996 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:03.996 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.996 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.996 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:03.997 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.997 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.997 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.997 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.997 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.997 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.997 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.998 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T10:41:03.998 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T10:41:03.998 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:03.998 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:03.998 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:03.998 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:03.998 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:03.998 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:03.999 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:03.999 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:03.999 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:03.999 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:03.999 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:03.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:03.999 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T10:41:04.000 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T10:41:04.000 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.000 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.000 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.000 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:04.000 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.000 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.000 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.000 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.001 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.001 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.001 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.001 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T10:41:04.001 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T10:41:04.001 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.001 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.002 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.002 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-04-21T10:41:04.002 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.002 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.002 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.002 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.002 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.002 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.003 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.003 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2024-04-21T10:41:04.003 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2024-04-21T10:41:04.003 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.003 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.003 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.003 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.004 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.004 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.004 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.004 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.004 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.004 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.004 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.005 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2024-04-21T10:41:04.005 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2024-04-21T10:41:04.005 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.005 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.005 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.005 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.005 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.005 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.006 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.006 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.006 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.006 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.006 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.006 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2024-04-21T10:41:04.007 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2024-04-21T10:41:04.007 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.007 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.007 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.007 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.007 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.007 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.007 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.008 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2024-04-21T10:41:04.008 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.008 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.008 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.008 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2024-04-21T10:41:04.008 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2024-04-21T10:41:04.008 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.009 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.009 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.009 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2024-04-21T10:41:04.009 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.009 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.009 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.009 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.010 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.010 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.010 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.010 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-04-21T10:41:04.010 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-04-21T10:41:04.010 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.010 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.011 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.011 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.011 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.011 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.011 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.011 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.011 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.012 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.012 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.012 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.012 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.012 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.012 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.012 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.013 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.013 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.013 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.013 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.013 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.013 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.013 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.013 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.014 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.014 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.014 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.014 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.014 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.014 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.014 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.015 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.015 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.015 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.015 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.015 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.015 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.015 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.015 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.016 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.016 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.016 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.016 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.016 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.016 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.016 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.017 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.017 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.017 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.017 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.017 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.017 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.017 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.017 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.018 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.018 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.018 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.018 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.018 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.018 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.018 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.018 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.019 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.019 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.019 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.019 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.019 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.019 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.019 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.020 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.020 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.020 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.020 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.020 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.020 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.020 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.020 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.021 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2024-04-21T10:41:04.021 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2024-04-21T10:41:04.021 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.021 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.021 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.021 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.022 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.022 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.022 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.022 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2024-04-21T10:41:04.022 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.023 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.023 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.023 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2024-04-21T10:41:04.023 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T10:41:04.023 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.024 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.024 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.024 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.024 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.024 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.024 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.024 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.024 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.024 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.025 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.025 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-04-21T10:41:04.025 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-04-21T10:41:04.025 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.025 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.025 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.025 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-04-21T10:41:04.025 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.026 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.026 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.026 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.026 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.026 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.026 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.026 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-04-21T10:41:04.027 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-04-21T10:41:04.027 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.027 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.027 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.027 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2024-04-21T10:41:04.027 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.027 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.027 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.027 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.028 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.028 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.028 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.028 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-04-21T10:41:04.028 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-04-21T10:41:04.028 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.028 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.029 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.029 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.029 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.029 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.029 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.029 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.029 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.029 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.030 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.030 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2024-04-21T10:41:04.030 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2024-04-21T10:41:04.030 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.030 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.030 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.030 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2024-04-21T10:41:04.031 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.031 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.031 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.031 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.031 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.031 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.031 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.032 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-04-21T10:41:04.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-04-21T10:41:04.032 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.032 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.032 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.032 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2024-04-21T10:41:04.032 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.032 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.033 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.033 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.033 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2024-04-21T10:41:04.033 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.033 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.033 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.033 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.034 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.034 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-04-21T10:41:04.034 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-04-21T10:41:04.034 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.034 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.034 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.034 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-04-21T10:41:04.035 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-04-21T10:41:04.035 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-04-21T10:41:04.035 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.035 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.035 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.035 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.035 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2024-04-21T10:41:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2024-04-21T10:41:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-04-21T10:41:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2024-04-21T10:41:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2024-04-21T10:41:04.039 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.039 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.039 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.039 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.039 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.039 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.039 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.039 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2024-04-21T10:41:04.040 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.040 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.040 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.040 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.040 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.040 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.040 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.040 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.041 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.041 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.041 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.041 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.041 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.041 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.041 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.041 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.042 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.042 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.042 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.042 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.042 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.042 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.042 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.042 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.043 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.043 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.043 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.043 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.043 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.043 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.043 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.043 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.043 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.044 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.044 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.044 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.044 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.044 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.044 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.044 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.045 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.045 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.045 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.045 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.045 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.045 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.045 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.045 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.046 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.046 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.046 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.046 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.046 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.046 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.046 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.046 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.047 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.047 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.047 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.047 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.047 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.047 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.048 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.048 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.048 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.048 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.048 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.048 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.048 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2024-04-21T10:41:04.048 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2024-04-21T10:41:04.049 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.049 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.049 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.049 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2024-04-21T10:41:04.049 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.049 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.049 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.049 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T10:41:04.049 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.050 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.050 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.050 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2024-04-21T10:41:04.050 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2024-04-21T10:41:04.050 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.050 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.050 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.050 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2024-04-21T10:41:04.051 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.051 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.051 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.051 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T10:41:04.051 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.051 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.051 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.051 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2024-04-21T10:41:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2024-04-21T10:41:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2024-04-21T10:41:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.052 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2024-04-21T10:41:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2024-04-21T10:41:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2024-04-21T10:41:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.053 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-04-21T10:41:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T10:41:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.054 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2024-04-21T10:41:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2024-04-21T10:41:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.055 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T10:41:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.056 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.057 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T10:41:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2024-04-21T10:41:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2024-04-21T10:41:04.058 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2024-04-21T10:41:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T10:41:04.059 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2024-04-21T10:41:04.060 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.061 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.062 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.063 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.063 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.063 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.063 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.063 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.063 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.063 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.063 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.064 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.064 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.064 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.064 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.064 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.064 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.064 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.065 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.065 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.065 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.065 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.065 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.065 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.066 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.066 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.066 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.066 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.066 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.066 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.066 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.066 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.067 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.067 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.067 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.067 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.067 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.067 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.067 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.067 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.067 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.068 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.068 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.068 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.068 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.068 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.068 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.069 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.069 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.069 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.069 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.069 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.069 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.069 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.069 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.070 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.070 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.070 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2024-04-21T10:41:04.070 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2024-04-21T10:41:04.070 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.070 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.071 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.071 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-04-21T10:41:04.071 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.071 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.071 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.071 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.071 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.071 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.071 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.072 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2024-04-21T10:41:04.072 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2024-04-21T10:41:04.072 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.072 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.072 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.072 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2024-04-21T10:41:04.072 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.073 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.073 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.073 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.073 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.073 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.073 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.073 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:04.074 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:04.074 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.074 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.074 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.074 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:04.074 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.074 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.074 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.075 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.075 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.075 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.075 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.075 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.075 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2024-04-21T10:41:04.076 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2024-04-21T10:41:04.076 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2024-04-21T10:41:04.076 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.076 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2024-04-21T10:41:04.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2024-04-21T10:41:04.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.077 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.077 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.077 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2024-04-21T10:41:04.077 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.077 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.077 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.077 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T10:41:04.078 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.078 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.078 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.078 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2024-04-21T10:41:04.078 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2024-04-21T10:41:04.078 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.079 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.079 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.079 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2024-04-21T10:41:04.079 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.079 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.079 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.079 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T10:41:04.079 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.080 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.080 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.080 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2024-04-21T10:41:04.080 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2024-04-21T10:41:04.080 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.080 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.081 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.081 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.081 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.081 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.081 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.081 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2024-04-21T10:41:04.081 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.081 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.082 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.082 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:04.082 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:04.082 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.082 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.082 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.082 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2024-04-21T10:41:04.083 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2024-04-21T10:41:04.083 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.083 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.083 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T10:41:04.083 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.083 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.083 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.084 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.084 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.084 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.084 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.084 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.084 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.085 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.085 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.085 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.085 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.085 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.085 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.085 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.085 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.086 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.086 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.086 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.086 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.086 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.086 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.087 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.087 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.087 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.087 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.087 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.087 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.087 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.087 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.087 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.088 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.088 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.088 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.088 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.088 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.088 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.088 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.089 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.089 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.089 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.089 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.089 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.089 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.089 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.089 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.090 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.090 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.090 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.090 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.090 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.090 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.090 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.091 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.091 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.091 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.091 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.091 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.091 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.091 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.092 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.092 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.092 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.092 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.092 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.092 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.092 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2024-04-21T10:41:04.093 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2024-04-21T10:41:04.093 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.093 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.093 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.093 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.093 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.093 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.094 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.094 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2024-04-21T10:41:04.094 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.094 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.094 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.094 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2024-04-21T10:41:04.094 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2024-04-21T10:41:04.095 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.095 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.095 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.095 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2024-04-21T10:41:04.095 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.095 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.095 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.095 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2024-04-21T10:41:04.096 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.096 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.096 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.096 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-04-21T10:41:04.096 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-04-21T10:41:04.096 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.096 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.097 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.097 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2024-04-21T10:41:04.097 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.097 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.097 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.097 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T10:41:04.097 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.097 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.098 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.098 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2024-04-21T10:41:04.098 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2024-04-21T10:41:04.098 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.098 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.098 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.098 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-04-21T10:41:04.099 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-04-21T10:41:04.099 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2024-04-21T10:41:04.099 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.099 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T10:41:04.099 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.099 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.099 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.100 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2024-04-21T10:41:04.100 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2024-04-21T10:41:04.100 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.100 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.100 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.100 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.100 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.100 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.100 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.101 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2024-04-21T10:41:04.101 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.101 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.101 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.101 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2024-04-21T10:41:04.101 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2024-04-21T10:41:04.101 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.102 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.102 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.102 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2024-04-21T10:41:04.102 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.102 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.102 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.102 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T10:41:04.102 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.103 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.103 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.103 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.103 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.103 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.103 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2024-04-21T10:41:04.103 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.103 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.104 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.104 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.104 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.104 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.104 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.104 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.104 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.104 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.104 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.105 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.105 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.105 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.105 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.105 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.105 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.105 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.105 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.106 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.106 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.106 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.106 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.106 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.106 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.106 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.107 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.107 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.107 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.107 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.107 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.107 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.107 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.107 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.108 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.108 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.108 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.108 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.108 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.108 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.108 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.109 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.109 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.109 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.109 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.109 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.109 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.109 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.110 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.110 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.110 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.110 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.110 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.110 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.110 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.111 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.111 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.111 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.111 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.111 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.111 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.112 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.112 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.112 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.112 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.112 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.112 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.112 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.113 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.113 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.113 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.113 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2024-04-21T10:41:04.113 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.114 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.114 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.114 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.114 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.114 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.114 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.114 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.115 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.115 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.115 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.115 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.115 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.115 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.115 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.115 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.116 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.116 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.116 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.116 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.116 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.116 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.116 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.117 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.117 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.117 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.117 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.117 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.117 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.117 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.118 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.118 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.118 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.118 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.118 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.118 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.119 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.119 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.119 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.119 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.119 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.119 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.119 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.120 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.120 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.120 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.120 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.120 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.120 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.120 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.120 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.120 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.121 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.121 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.121 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.121 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.121 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.121 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.121 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.122 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.122 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.122 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.122 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.122 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.122 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.122 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.122 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.123 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.123 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.123 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.123 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.123 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2024-04-21T10:41:04.123 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.123 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.124 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.124 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2024-04-21T10:41:04.124 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2024-04-21T10:41:04.124 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.124 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.124 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.124 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2024-04-21T10:41:04.124 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.124 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.125 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.125 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T10:41:04.125 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.125 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.125 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.125 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2024-04-21T10:41:04.125 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2024-04-21T10:41:04.126 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.126 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.126 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.126 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:04.126 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2024-04-21T10:41:04.126 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.126 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.126 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T10:41:04.126 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.127 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.127 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.127 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.127 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.127 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.127 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.127 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.128 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.128 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.128 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.128 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.128 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.128 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.128 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.128 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.128 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.129 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.129 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.129 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.129 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.129 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.129 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.129 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.130 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.130 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.130 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.130 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.130 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.130 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.130 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.130 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.131 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.131 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.131 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.131 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.131 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.131 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.131 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.132 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.132 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.132 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.132 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.132 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.132 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.132 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.132 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.133 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.133 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.133 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.133 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.133 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.133 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.133 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.133 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.134 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.134 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.134 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.134 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.134 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.134 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.134 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.134 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.134 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.135 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.135 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.135 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.135 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.135 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.135 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.135 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.136 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2024-04-21T10:41:04.136 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.136 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.136 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.136 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2024-04-21T10:41:04.136 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2024-04-21T10:41:04.136 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.136 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.137 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.137 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2024-04-21T10:41:04.137 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.137 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.137 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.137 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2024-04-21T10:41:04.137 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T10:41:04.137 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.137 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.138 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.138 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.138 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.138 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.138 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.138 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.138 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.139 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.139 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.139 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.139 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.139 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.139 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.139 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.139 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.140 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.140 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.140 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.140 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.140 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.141 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.141 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.141 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.141 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.141 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.141 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.141 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.141 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.142 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.142 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.142 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.142 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.142 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.142 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.142 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.143 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.143 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.143 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.143 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.143 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.143 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.143 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.143 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.144 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.144 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.144 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.144 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.144 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.144 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.144 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.144 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.145 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.145 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.145 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.145 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.145 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.145 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.145 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.146 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.146 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.146 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.146 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.146 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.146 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.146 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.147 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.147 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2024-04-21T10:41:04.147 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2024-04-21T10:41:04.147 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.147 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.147 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.148 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.148 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.148 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.148 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.148 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2024-04-21T10:41:04.148 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T10:41:04.148 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.149 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.149 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.149 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2024-04-21T10:41:04.149 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2024-04-21T10:41:04.149 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.149 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.149 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.149 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2024-04-21T10:41:04.150 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.150 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.150 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.150 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2024-04-21T10:41:04.150 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.150 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.151 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.151 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2024-04-21T10:41:04.151 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2024-04-21T10:41:04.151 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.151 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.151 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.152 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2024-04-21T10:41:04.152 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.152 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.152 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.152 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T10:41:04.152 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.152 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.152 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.153 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2024-04-21T10:41:04.153 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2024-04-21T10:41:04.153 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.153 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.153 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.153 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.154 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.154 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.154 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.154 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2024-04-21T10:41:04.154 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.154 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.154 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.155 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2024-04-21T10:41:04.155 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2024-04-21T10:41:04.155 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.155 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.155 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.155 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2024-04-21T10:41:04.155 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.155 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.156 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.156 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T10:41:04.156 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T10:41:04.156 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.156 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.156 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.156 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.157 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.157 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2024-04-21T10:41:04.157 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.157 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.157 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.157 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.157 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.158 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.158 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.158 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.158 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.158 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.161 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.161 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.161 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.162 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.162 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.162 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.162 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.162 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.162 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.162 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.162 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.163 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.163 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.163 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.163 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.163 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.163 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.163 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.164 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.164 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.164 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.164 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.164 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.164 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.164 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.164 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.164 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.165 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.165 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.165 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.165 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.165 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.165 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.165 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.165 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.166 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.166 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.166 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.166 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.166 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.166 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.166 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.166 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.167 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.167 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.167 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.167 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.167 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.167 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.167 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.167 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.168 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.168 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.168 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.168 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.168 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.168 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.168 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.168 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.169 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.169 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.169 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.169 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.169 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.169 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2024-04-21T10:41:04.169 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.170 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.170 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.170 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.170 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.170 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.170 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.171 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.171 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.171 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.171 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.171 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.171 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.172 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.172 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.172 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.172 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.172 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.172 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.173 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.173 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.173 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.173 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.173 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.173 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.173 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.174 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.174 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.174 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.174 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.174 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.174 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.175 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.175 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.175 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.175 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.175 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.175 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.176 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.176 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.176 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.176 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.176 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.176 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.176 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.177 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.177 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.177 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.177 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.177 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.177 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.177 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.178 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.178 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.178 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.178 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.178 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.178 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.179 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.179 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.179 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.179 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.179 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.179 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.179 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.179 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.180 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.180 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.180 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.180 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.180 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2024-04-21T10:41:04.180 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.180 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.181 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.181 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.181 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.181 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.181 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.181 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.182 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.182 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.182 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.182 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.182 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.182 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.182 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.183 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.183 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.183 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.183 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.183 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.183 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.183 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.183 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.184 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.184 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.184 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.184 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.184 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.184 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.184 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.184 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.185 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.185 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.185 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.185 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.185 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.185 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.185 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.185 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.185 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.186 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.186 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.186 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.186 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.186 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.186 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.186 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.186 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.187 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.187 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.187 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.187 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.187 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.187 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.187 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.187 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.188 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.188 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.188 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.188 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.188 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.188 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.188 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.188 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.189 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.189 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.189 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.189 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.189 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.189 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.189 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.189 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.190 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-04-21T10:41:04.190 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.190 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.190 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.190 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2024-04-21T10:41:04.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2024-04-21T10:41:04.191 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.191 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.191 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.191 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.191 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.191 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.191 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.192 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T10:41:04.192 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.192 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.192 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.192 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.192 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.192 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.192 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.192 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.193 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.193 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.193 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.193 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.193 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.193 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.193 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.194 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.194 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.194 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.194 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.194 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.194 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.195 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.195 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.195 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.195 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.195 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.195 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.196 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.196 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.196 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.196 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.196 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.196 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.196 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.196 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.197 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.197 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.197 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.197 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.197 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.197 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.197 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.197 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.198 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.198 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.198 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.198 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.198 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.198 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.198 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.198 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.199 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.199 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.199 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.199 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.199 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.199 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.199 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.200 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.200 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.200 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.200 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.200 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.200 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.200 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.201 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.201 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.201 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.201 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2024-04-21T10:41:04.201 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-04-21T10:41:04.201 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.202 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.202 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.202 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.202 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.202 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.202 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.203 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-04-21T10:41:04.203 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2024-04-21T10:41:04.203 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2024-04-21T10:41:04.203 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.203 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2024-04-21T10:41:04.203 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.203 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.203 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.204 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.204 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.204 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.204 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.204 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2024-04-21T10:41:04.204 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.204 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.205 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.205 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.205 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.205 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.205 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.205 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.205 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.206 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.206 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.206 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.206 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.206 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.206 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.206 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.206 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.207 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.207 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.207 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.207 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.207 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.207 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.208 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.208 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.208 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.208 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.208 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.208 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.208 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.208 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.209 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.209 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.209 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.209 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.209 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.209 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.210 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.210 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.210 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.210 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.210 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.210 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.210 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.211 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.211 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.211 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.211 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.211 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.211 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.211 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.212 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.212 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.212 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.212 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.212 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.212 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.213 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.213 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.213 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.213 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.213 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.213 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.213 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.213 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.214 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.214 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.214 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.214 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.214 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.214 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2024-04-21T10:41:04.215 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.215 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.215 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.215 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.215 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.215 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.216 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.216 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.216 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.216 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.216 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.216 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.216 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.217 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.217 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.217 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.217 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.217 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.217 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.217 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.217 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.218 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.218 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.218 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.218 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.218 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.219 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.219 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.219 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.219 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.219 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.219 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.219 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.219 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.220 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.220 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.220 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.220 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.220 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.220 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.221 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.221 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.221 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.221 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.221 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.221 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.221 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.221 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.222 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.222 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.222 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.222 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.222 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.222 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.222 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.223 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.223 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.223 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.223 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.223 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.223 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.223 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.223 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.224 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.224 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.224 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.224 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.224 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.224 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.224 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.224 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.225 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.225 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.225 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2024-04-21T10:41:04.225 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.225 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.225 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.225 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.226 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.226 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.226 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.226 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.226 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.226 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.226 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.227 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.227 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.227 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.227 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.227 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.227 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.227 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.227 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.228 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.228 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.228 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.228 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.228 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.228 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.228 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.228 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.229 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.229 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.229 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.229 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.229 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.229 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.229 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.230 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.230 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.230 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.230 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.230 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.230 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.230 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.231 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.231 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.231 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.231 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.231 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.231 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.231 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.231 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.232 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.232 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.232 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.232 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.232 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.232 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.232 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.232 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.233 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.233 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.233 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.233 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.233 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.233 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.233 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.233 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.234 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.234 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.234 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.234 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.234 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.234 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.234 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.235 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.235 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:04.235 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:04.235 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.235 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.235 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.235 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:04.235 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.236 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.236 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.236 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.236 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.236 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.236 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.237 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2024-04-21T10:41:04.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2024-04-21T10:41:04.237 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:04.237 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.237 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.237 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-04-21T10:41:04.237 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2024-04-21T10:41:04.238 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.238 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.238 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2024-04-21T10:41:04.238 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T10:41:04.238 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.238 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.239 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.239 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.239 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2024-04-21T10:41:04.239 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.239 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.239 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.239 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2024-04-21T10:41:04.239 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2024-04-21T10:41:04.240 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.240 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.240 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.240 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.240 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.240 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.240 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.241 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T10:41:04.241 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.241 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.241 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.241 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-04-21T10:41:04.241 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-04-21T10:41:04.242 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.242 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.242 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.242 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.242 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.242 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.242 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.243 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.243 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.243 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.243 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.243 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.243 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.243 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.244 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.244 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.244 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.244 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.244 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.244 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.245 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.245 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.245 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.245 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.245 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.245 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.245 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.245 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.246 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.246 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.246 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.247 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.247 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.247 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.247 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.247 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.247 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.247 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.247 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.248 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.248 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.248 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.249 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.249 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.249 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.249 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.249 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.249 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.249 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.250 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.250 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.250 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.250 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.250 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.250 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.250 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.251 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.251 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.251 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.251 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.251 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.251 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.251 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.252 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.252 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.252 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.252 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.252 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.252 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.252 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.252 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.253 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.253 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2024-04-21T10:41:04.253 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2024-04-21T10:41:04.253 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.253 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.253 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.253 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2024-04-21T10:41:04.254 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.254 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.254 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.254 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2024-04-21T10:41:04.254 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.254 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.254 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.255 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:04.255 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:04.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.255 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.255 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.255 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-04-21T10:41:04.256 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.256 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.256 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.256 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2024-04-21T10:41:04.256 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.256 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.256 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.256 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.257 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2024-04-21T10:41:04.257 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.257 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.257 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.258 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2024-04-21T10:41:04.258 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2024-04-21T10:41:04.258 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.258 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.258 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.258 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.258 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.259 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.259 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.259 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.259 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.259 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.259 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.259 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2024-04-21T10:41:04.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2024-04-21T10:41:04.260 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.260 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.260 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.260 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.260 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.260 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.260 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.261 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T10:41:04.261 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T10:41:04.261 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.261 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.261 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.261 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.262 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.262 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.262 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.262 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.262 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.262 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.262 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.262 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.263 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.263 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.263 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.263 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.263 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.263 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.263 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.263 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.263 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.264 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.264 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.264 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.264 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.264 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.264 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.265 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.265 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.265 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.265 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.265 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.265 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.265 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.265 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.266 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.266 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.266 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.266 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.266 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.266 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.266 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.266 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.267 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.267 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.267 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.267 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.267 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.267 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.267 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.267 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.268 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.268 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.268 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.268 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.268 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.268 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.268 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.269 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.269 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.269 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.269 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.269 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.269 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.269 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.269 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.269 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.270 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.270 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2024-04-21T10:41:04.270 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2024-04-21T10:41:04.270 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.270 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.270 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.271 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.271 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.271 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.271 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.271 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.271 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.271 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.271 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.272 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T10:41:04.272 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T10:41:04.272 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.272 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.272 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.272 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2024-04-21T10:41:04.272 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.272 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.273 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.273 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.273 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.273 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.273 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.273 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2024-04-21T10:41:04.273 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2024-04-21T10:41:04.274 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:04.274 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.274 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.274 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2024-04-21T10:41:04.274 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.274 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.274 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.274 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.275 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.275 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.275 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.275 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-04-21T10:41:04.275 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-04-21T10:41:04.275 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.275 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.275 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.276 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-04-21T10:41:04.276 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.276 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.276 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.276 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T10:41:04.276 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.276 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.276 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.277 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-04-21T10:41:04.277 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-04-21T10:41:04.277 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.277 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.277 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.277 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2024-04-21T10:41:04.277 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.278 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.278 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.278 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T10:41:04.278 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.278 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.278 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.278 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2024-04-21T10:41:04.278 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2024-04-21T10:41:04.279 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.279 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.279 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.279 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2024-04-21T10:41:04.279 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.279 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.279 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.279 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.280 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.280 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.280 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.280 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-04-21T10:41:04.280 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-04-21T10:41:04.280 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.280 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.281 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.281 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2024-04-21T10:41:04.281 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.281 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.281 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.281 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2024-04-21T10:41:04.281 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2024-04-21T10:41:04.281 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.281 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.282 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.282 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.282 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.282 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-04-21T10:41:04.282 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-04-21T10:41:04.282 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.282 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.283 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.283 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-04-21T10:41:04.283 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-04-21T10:41:04.283 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-04-21T10:41:04.283 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.283 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.283 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.283 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.283 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.284 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.284 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.284 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.284 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-04-21T10:41:04.284 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.284 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.284 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.284 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.284 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.285 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.285 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.285 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.285 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.285 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.285 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.285 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.285 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.285 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.286 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.286 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.286 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.286 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.286 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.286 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.286 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.286 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.287 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.287 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.287 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.287 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.287 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.287 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.288 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.288 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.288 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.288 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.288 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.288 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.288 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.288 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.289 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.289 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.289 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.289 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.289 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.289 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.289 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.289 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.290 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.290 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.290 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.290 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.290 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.290 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.290 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.290 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.291 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.291 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.291 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.291 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.291 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.291 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.291 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.292 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.292 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.292 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.292 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.292 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.292 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.292 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.292 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.292 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.292 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.293 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2024-04-21T10:41:04.293 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2024-04-21T10:41:04.293 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.293 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.293 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.293 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:04.293 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.294 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.294 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.294 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.294 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.294 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.294 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.294 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2024-04-21T10:41:04.295 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T10:41:04.295 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.295 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.295 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.295 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.295 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.295 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.295 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.296 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.296 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.296 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.296 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.296 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2024-04-21T10:41:04.296 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2024-04-21T10:41:04.297 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.297 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.297 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.297 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.297 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.297 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.297 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.298 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.298 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.298 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.298 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.298 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.298 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2024-04-21T10:41:04.299 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.299 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.299 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.299 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2024-04-21T10:41:04.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2024-04-21T10:41:04.300 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.300 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.300 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.300 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.300 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.300 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.300 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.300 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.301 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.301 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.301 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.301 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-04-21T10:41:04.301 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-04-21T10:41:04.301 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.302 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.302 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.302 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.302 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.302 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.302 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.302 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.303 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.303 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.303 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.303 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.303 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.303 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.303 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.304 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.304 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.304 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.304 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.304 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.304 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.304 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.304 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.305 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.305 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.305 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.305 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.305 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.305 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.305 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.305 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.306 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.306 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.306 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.306 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.306 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.306 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.307 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.307 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.307 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.307 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.307 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.307 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.307 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.308 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.308 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.308 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.308 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.308 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.308 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.308 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.309 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.309 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.309 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.309 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.309 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.309 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.309 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.309 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.310 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.310 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.310 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.310 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.310 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.311 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.311 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.311 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.311 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.311 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.311 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.312 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-04-21T10:41:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-04-21T10:41:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.313 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.314 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-04-21T10:41:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-04-21T10:41:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.315 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2024-04-21T10:41:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.317 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.318 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.321 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.322 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.323 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.324 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.325 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.326 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2024-04-21T10:41:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.327 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.328 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.329 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.330 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.331 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.332 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.333 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.335 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.336 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2024-04-21T10:41:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2024-04-21T10:41:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2024-04-21T10:41:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.337 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T10:41:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2024-04-21T10:41:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.339 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.340 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2024-04-21T10:41:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2024-04-21T10:41:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.349 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2024-04-21T10:41:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2024-04-21T10:41:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2024-04-21T10:41:04.350 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.351 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2024-04-21T10:41:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2024-04-21T10:41:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.352 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2024-04-21T10:41:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2024-04-21T10:41:04.353 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.354 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.354 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.354 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.354 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.354 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.354 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.354 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.355 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.355 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.355 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.355 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2024-04-21T10:41:04.355 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2024-04-21T10:41:04.355 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:04.355 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.356 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.356 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.356 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.356 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.356 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.356 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.356 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.356 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.357 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.357 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2024-04-21T10:41:04.357 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2024-04-21T10:41:04.357 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.357 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.357 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.358 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.358 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.358 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.358 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.358 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.358 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.358 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.358 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.359 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2024-04-21T10:41:04.359 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2024-04-21T10:41:04.359 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.359 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.359 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.359 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.360 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.360 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.360 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.360 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.360 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.360 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.360 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.361 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2024-04-21T10:41:04.361 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2024-04-21T10:41:04.361 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.361 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.361 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.361 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.361 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.361 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.362 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.362 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.362 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.362 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.362 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.363 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2024-04-21T10:41:04.363 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2024-04-21T10:41:04.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.363 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.363 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.363 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.365 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-04-21T10:41:04.365 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2024-04-21T10:41:04.365 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.365 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.366 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.366 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.366 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.366 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2024-04-21T10:41:04.367 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2024-04-21T10:41:04.367 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.367 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.367 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.367 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.368 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.368 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.368 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.368 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.368 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.369 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.369 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.369 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2024-04-21T10:41:04.369 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2024-04-21T10:41:04.370 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.370 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.370 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.370 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.370 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.371 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.371 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.371 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.371 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.371 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.372 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.372 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2024-04-21T10:41:04.372 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2024-04-21T10:41:04.372 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.373 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.373 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.373 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.373 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.373 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.374 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.374 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.374 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.374 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.374 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.375 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.375 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.375 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-04-21T10:41:04.376 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.376 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.376 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.376 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2024-04-21T10:41:04.376 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2024-04-21T10:41:04.376 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.377 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.377 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.377 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.377 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.377 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.377 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.378 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T10:41:04.378 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.378 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.378 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.378 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2024-04-21T10:41:04.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2024-04-21T10:41:04.378 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.379 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.379 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.379 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.379 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.379 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.379 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.379 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2024-04-21T10:41:04.379 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.380 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.380 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.380 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.380 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.380 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.380 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.380 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.381 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.381 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.381 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.381 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.381 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.381 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.381 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.381 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.381 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.382 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.382 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.382 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.382 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.382 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.382 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.383 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.383 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.383 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.383 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.383 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.383 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.383 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.383 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.383 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.384 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.384 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.384 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.384 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.384 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.384 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.385 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.385 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.385 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.385 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.385 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.385 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.385 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.385 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.385 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.386 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.386 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.386 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.386 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.386 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.386 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.386 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.387 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.387 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.387 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.387 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.387 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.387 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.387 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.388 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.388 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.388 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.388 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.388 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.388 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.388 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.389 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.389 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.389 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.389 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2024-04-21T10:41:04.389 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.389 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.390 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.390 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.390 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.390 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.390 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.390 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.390 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.391 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.391 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.391 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.391 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.391 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.391 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.391 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.391 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.392 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.392 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.392 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.392 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.392 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.392 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.392 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.393 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.393 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.393 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.393 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.393 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.393 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.393 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.393 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.394 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.394 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.394 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.394 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.394 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.394 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.394 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.395 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.395 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.395 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.395 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.395 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.395 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.395 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.395 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.395 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.396 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.396 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.396 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.396 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.396 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.396 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.396 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.396 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.397 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.397 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.397 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.397 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.397 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.397 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.397 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.397 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.398 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.398 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.398 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.398 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.398 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.398 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.398 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.399 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2024-04-21T10:41:04.399 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.399 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.399 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.399 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.399 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.399 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.399 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.400 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.400 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.400 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.400 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.400 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.400 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.400 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.401 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.401 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.401 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.401 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.401 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.401 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.401 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.401 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.402 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.402 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.402 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.402 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.402 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.402 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.402 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.402 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.403 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.403 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.403 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.403 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.403 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.403 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.403 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.404 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.404 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.404 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.404 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.404 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.404 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.404 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.404 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.405 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.405 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.405 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.405 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.405 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.405 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.405 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.405 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.405 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.406 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.406 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.406 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.406 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.406 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.406 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.407 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.407 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.407 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.407 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.407 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.407 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.407 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.407 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.407 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.408 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.408 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.408 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2024-04-21T10:41:04.408 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.408 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.408 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.409 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2024-04-21T10:41:04.409 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2024-04-21T10:41:04.409 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.409 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.409 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.409 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T10:41:04.409 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.409 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.409 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.410 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.410 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.410 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.410 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.410 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:04.410 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:04.410 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.411 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.411 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.411 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2024-04-21T10:41:04.411 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.411 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.411 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.411 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.411 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.412 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.412 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.412 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.412 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.412 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.412 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.412 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.413 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.413 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.413 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.413 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.413 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.413 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.413 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.413 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.414 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.414 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.414 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.414 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.414 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.414 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.414 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.415 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.415 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.415 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.415 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.415 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.415 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.415 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.415 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.416 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.416 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.416 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.416 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.416 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.416 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.416 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.417 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.417 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.417 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.417 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.417 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.417 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.417 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.417 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.418 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.418 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.418 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.418 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.418 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.418 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.418 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.418 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.419 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.419 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.419 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.419 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.419 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.420 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.420 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.420 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.420 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.420 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.420 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.420 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.421 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.421 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.421 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.421 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.421 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.421 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.421 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2024-04-21T10:41:04.422 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.422 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.422 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.422 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2024-04-21T10:41:04.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2024-04-21T10:41:04.422 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.422 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.423 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.423 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.423 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.423 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.423 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.423 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T10:41:04.423 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.424 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.424 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.424 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2024-04-21T10:41:04.424 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2024-04-21T10:41:04.424 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.424 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.424 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.425 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.425 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.425 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.425 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.425 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T10:41:04.425 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.426 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.426 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.426 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2024-04-21T10:41:04.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2024-04-21T10:41:04.426 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.426 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.427 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.427 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.427 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.427 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.427 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.427 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T10:41:04.427 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.427 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.428 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.428 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2024-04-21T10:41:04.428 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2024-04-21T10:41:04.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.428 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.428 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.428 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.429 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.429 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.429 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.429 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T10:41:04.429 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.429 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.429 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.430 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2024-04-21T10:41:04.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2024-04-21T10:41:04.430 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.430 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.430 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.430 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.431 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.431 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.431 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.431 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T10:41:04.431 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.431 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.431 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.432 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2024-04-21T10:41:04.432 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2024-04-21T10:41:04.432 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.432 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.432 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.432 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.433 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.433 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.433 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.433 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.433 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.433 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.433 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.434 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2024-04-21T10:41:04.434 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2024-04-21T10:41:04.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.434 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.434 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.434 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.434 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.435 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.435 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.435 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.435 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.435 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.435 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.435 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2024-04-21T10:41:04.436 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2024-04-21T10:41:04.436 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.436 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.436 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.436 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T10:41:04.436 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.436 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.437 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.437 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.437 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.437 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.437 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.437 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-04-21T10:41:04.437 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-04-21T10:41:04.438 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.438 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.438 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.438 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.438 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.438 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.438 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.439 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.439 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.439 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.439 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.439 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:04.439 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:04.439 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.440 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.440 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.440 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2024-04-21T10:41:04.440 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2024-04-21T10:41:04.440 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.440 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.440 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.441 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.441 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.441 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.441 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2024-04-21T10:41:04.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2024-04-21T10:41:04.441 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.441 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.442 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.442 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2024-04-21T10:41:04.442 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.442 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.442 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.442 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.442 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.443 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.443 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.443 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2024-04-21T10:41:04.443 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2024-04-21T10:41:04.443 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.443 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.443 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.444 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.444 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.444 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.444 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.444 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.444 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.444 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.444 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.445 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2024-04-21T10:41:04.445 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2024-04-21T10:41:04.445 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.445 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.445 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.445 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.445 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.446 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.446 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.446 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.446 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.446 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.446 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.446 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.447 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.447 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.447 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.447 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.447 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.447 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.447 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.447 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.448 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.448 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.448 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.448 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.448 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.448 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.448 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.448 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.449 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.449 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.449 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.449 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.449 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.449 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.449 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.450 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.450 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.450 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.450 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.450 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.450 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.450 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.450 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.451 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.451 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.451 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.451 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.451 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.451 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.451 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.452 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.452 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.452 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.452 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.452 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.452 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.452 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.453 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.453 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.453 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.453 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.453 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.453 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.453 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.453 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.454 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.454 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.454 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.454 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.454 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.454 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.454 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.455 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.455 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.455 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.455 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.455 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.455 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.456 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2024-04-21T10:41:04.456 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2024-04-21T10:41:04.456 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.456 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.456 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.456 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.456 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.456 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.457 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.457 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.457 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.457 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.457 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.457 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2024-04-21T10:41:04.457 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2024-04-21T10:41:04.458 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.458 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.458 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.458 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.458 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.458 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.458 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.458 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.459 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.459 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.459 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.459 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2024-04-21T10:41:04.459 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2024-04-21T10:41:04.459 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.459 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.460 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.460 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T10:41:04.460 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.460 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.460 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.460 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.460 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.460 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.461 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.461 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.461 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.461 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.461 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.461 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2024-04-21T10:41:04.461 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.461 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.467 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.467 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.467 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.467 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.467 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.467 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.467 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.467 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-04-21T10:41:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2024-04-21T10:41:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2024-04-21T10:41:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2024-04-21T10:41:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.482 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T10:41:04.482 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.482 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.482 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.482 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2024-04-21T10:41:04.482 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2024-04-21T10:41:04.482 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.483 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.483 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.483 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.483 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.483 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.483 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.483 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2024-04-21T10:41:04.483 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.484 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.484 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.484 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2024-04-21T10:41:04.484 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2024-04-21T10:41:04.484 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.484 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.485 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.485 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:04.485 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.485 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.485 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.485 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T10:41:04.485 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.486 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.486 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.486 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_no_wait": { 2024-04-21T10:41:04.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T10:41:04.486 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.486 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.486 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.487 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.487 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.487 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.487 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.487 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T10:41:04.487 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.487 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.487 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.487 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.488 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.488 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2024-04-21T10:41:04.488 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.488 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.488 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.488 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2024-04-21T10:41:04.489 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2024-04-21T10:41:04.489 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:04.489 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.489 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.489 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2024-04-21T10:41:04.489 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.489 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.489 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.490 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.490 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.490 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.490 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.490 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2024-04-21T10:41:04.491 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2024-04-21T10:41:04.491 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.491 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.491 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.491 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.491 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.491 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.491 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.492 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.492 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.492 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.492 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.492 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:04.492 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:04.492 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.493 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.493 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.493 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:04.493 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.493 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.493 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.493 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.494 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.494 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.494 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.494 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.494 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.494 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.494 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.495 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.495 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.495 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.495 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.495 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.495 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.495 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.496 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.496 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.496 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.496 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.496 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.496 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.496 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.496 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.496 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.497 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.497 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.497 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.497 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.497 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.497 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.497 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.498 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.498 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.498 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.498 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.498 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.498 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.500 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.500 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.500 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.500 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.500 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.500 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.500 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.500 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.501 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.501 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.501 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.501 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.501 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.501 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.501 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.502 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.502 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.502 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.502 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.502 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.502 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.502 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.502 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.503 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.503 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.503 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2024-04-21T10:41:04.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2024-04-21T10:41:04.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.503 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.503 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.504 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.504 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.504 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.504 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.504 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.504 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.504 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.504 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.505 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2024-04-21T10:41:04.505 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2024-04-21T10:41:04.505 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.505 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.505 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.505 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2024-04-21T10:41:04.505 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.506 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.506 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.506 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.506 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.506 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.506 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.506 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2024-04-21T10:41:04.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2024-04-21T10:41:04.507 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.507 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.507 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.507 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T10:41:04.507 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.507 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.507 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.508 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.508 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.508 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.508 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.508 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.508 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.508 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.508 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-21T10:41:04.509 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.509 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.509 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-04-21T10:41:04.509 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-04-21T10:41:04.509 INFO:teuthology.orchestra.run.smithi026.stdout: "dashboard", 2024-04-21T10:41:04.509 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-04-21T10:41:04.509 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-04-21T10:41:04.509 INFO:teuthology.orchestra.run.smithi026.stdout: "restful", 2024-04-21T10:41:04.510 INFO:teuthology.orchestra.run.smithi026.stdout: "snap_schedule" 2024-04-21T10:41:04.510 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.510 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2024-04-21T10:41:04.510 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.510 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2024-04-21T10:41:04.510 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.510 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.510 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.511 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:04.511 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:04.511 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.511 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.511 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.511 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:04.511 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.511 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.512 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.512 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T10:41:04.512 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.512 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.512 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.512 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.512 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.512 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.513 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.513 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.513 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.513 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.513 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.513 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.513 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.513 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.514 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.514 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.514 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.514 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.514 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.514 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.514 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.514 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.515 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.515 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.515 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.515 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.515 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.515 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.515 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.515 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.516 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.516 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.516 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.516 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.516 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.516 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.516 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.517 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.517 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.517 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.517 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.517 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.517 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.517 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.518 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.518 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.518 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.518 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.518 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.518 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.518 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.519 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.519 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.519 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.519 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.519 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.519 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.519 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.520 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.520 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.520 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.520 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.520 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.520 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.520 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.521 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.521 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.521 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.521 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.521 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.521 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.521 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.521 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2024-04-21T10:41:04.522 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2024-04-21T10:41:04.522 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.522 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.522 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.522 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.522 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.522 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.523 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.523 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2024-04-21T10:41:04.523 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.523 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.523 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.523 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.523 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2024-04-21T10:41:04.523 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2024-04-21T10:41:04.524 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.524 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.524 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.524 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2024-04-21T10:41:04.524 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.524 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.524 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.525 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2024-04-21T10:41:04.525 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.525 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.525 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.525 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.525 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2024-04-21T10:41:04.525 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2024-04-21T10:41:04.525 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.526 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.526 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.526 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.526 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.526 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.526 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.526 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2024-04-21T10:41:04.527 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.527 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.527 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.527 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.527 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2024-04-21T10:41:04.527 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2024-04-21T10:41:04.527 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.527 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.528 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.528 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.528 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.528 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.528 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.528 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2024-04-21T10:41:04.528 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.528 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.529 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.529 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.529 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2024-04-21T10:41:04.529 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2024-04-21T10:41:04.529 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.529 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.529 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2024-04-21T10:41:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2024-04-21T10:41:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2024-04-21T10:41:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2024-04-21T10:41:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2024-04-21T10:41:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2024-04-21T10:41:04.533 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2024-04-21T10:41:04.533 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.533 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.533 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.533 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.533 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.533 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.533 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T10:41:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2024-04-21T10:41:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2024-04-21T10:41:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.534 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.535 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.535 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.535 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.535 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.535 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.535 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2024-04-21T10:41:04.535 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.535 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.536 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.536 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.536 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.536 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.536 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2024-04-21T10:41:04.536 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.536 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.536 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.537 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2024-04-21T10:41:04.537 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2024-04-21T10:41:04.537 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.537 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.537 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.537 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.537 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.538 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.538 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T10:41:04.538 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.538 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.538 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.538 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.538 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2024-04-21T10:41:04.539 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2024-04-21T10:41:04.539 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.539 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.539 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.539 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2024-04-21T10:41:04.539 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.539 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.539 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.539 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T10:41:04.540 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T10:41:04.540 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.540 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.540 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2024-04-21T10:41:04.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2024-04-21T10:41:04.540 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:04.540 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.541 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.541 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:04.541 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-04-21T10:41:04.541 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-04-21T10:41:04.541 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.541 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T10:41:04.541 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T10:41:04.541 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.541 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.542 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2024-04-21T10:41:04.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2024-04-21T10:41:04.542 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:04.542 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.542 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.542 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2024-04-21T10:41:04.543 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-04-21T10:41:04.543 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2024-04-21T10:41:04.543 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.543 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T10:41:04.543 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.543 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.543 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.544 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2024-04-21T10:41:04.544 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2024-04-21T10:41:04.544 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.544 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.544 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.544 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T10:41:04.544 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.545 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.545 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.545 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T10:41:04.545 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T10:41:04.545 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.545 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.545 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2024-04-21T10:41:04.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2024-04-21T10:41:04.546 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:04.546 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.546 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.546 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2024-04-21T10:41:04.546 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2024-04-21T10:41:04.547 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2024-04-21T10:41:04.547 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.547 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2024-04-21T10:41:04.547 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T10:41:04.547 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.547 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.548 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2024-04-21T10:41:04.548 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2024-04-21T10:41:04.548 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.548 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.548 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.548 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2024-04-21T10:41:04.548 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.548 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.549 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.549 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T10:41:04.549 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T10:41:04.549 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.549 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.549 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2024-04-21T10:41:04.549 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2024-04-21T10:41:04.550 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:04.550 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.550 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.550 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:04.550 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2024-04-21T10:41:04.550 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2024-04-21T10:41:04.550 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.550 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T10:41:04.551 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T10:41:04.551 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.551 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.551 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.551 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.551 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.551 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.552 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.552 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.552 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.552 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.552 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.552 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.552 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.552 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.553 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.553 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.553 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.553 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.553 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.553 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.553 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.553 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.554 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.554 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.554 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.554 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.554 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.554 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.554 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.555 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.555 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.555 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.555 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.555 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.556 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.556 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.556 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.556 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.556 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.556 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.556 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.557 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.557 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.557 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.557 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.557 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.557 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.557 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.557 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.558 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.558 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.558 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.558 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.558 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.558 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.559 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.559 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.559 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.559 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.559 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.559 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.559 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.559 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.560 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.560 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2024-04-21T10:41:04.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2024-04-21T10:41:04.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:04.560 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.560 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.560 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:04.561 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.561 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.561 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.561 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T10:41:04.561 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.561 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.561 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.562 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-04-21T10:41:04.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-04-21T10:41:04.562 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.562 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.562 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.562 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2024-04-21T10:41:04.562 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.563 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.563 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.563 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2024-04-21T10:41:04.563 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2024-04-21T10:41:04.563 INFO:teuthology.orchestra.run.smithi026.stdout: "read", 2024-04-21T10:41:04.563 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap", 2024-04-21T10:41:04.563 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap-read" 2024-04-21T10:41:04.564 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.564 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2024-04-21T10:41:04.564 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.564 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.564 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.564 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2024-04-21T10:41:04.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2024-04-21T10:41:04.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.565 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.565 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.565 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.565 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.565 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.565 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.565 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T10:41:04.566 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.566 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.566 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.566 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:04.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:04.566 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.566 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.569 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.569 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:04.569 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.569 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.569 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.569 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T10:41:04.570 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.570 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.570 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.570 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2024-04-21T10:41:04.570 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2024-04-21T10:41:04.570 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.570 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.571 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.571 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-04-21T10:41:04.571 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-04-21T10:41:04.571 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.571 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.571 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T10:41:04.571 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T10:41:04.571 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.571 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2024-04-21T10:41:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2024-04-21T10:41:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:04.572 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.573 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.573 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.573 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T10:41:04.573 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.573 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.573 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.573 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.573 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.574 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.574 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2024-04-21T10:41:04.574 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.574 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.574 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.574 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2024-04-21T10:41:04.574 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2024-04-21T10:41:04.575 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:04.575 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.575 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.575 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-04-21T10:41:04.575 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.575 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.575 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.575 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T10:41:04.576 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.576 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.576 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.576 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2024-04-21T10:41:04.576 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2024-04-21T10:41:04.576 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.577 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.577 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.577 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2024-04-21T10:41:04.577 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.577 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.577 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.577 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T10:41:04.578 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.578 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.578 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.578 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2024-04-21T10:41:04.578 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2024-04-21T10:41:04.578 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.579 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.579 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.579 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2024-04-21T10:41:04.579 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.579 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.579 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.579 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T10:41:04.580 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.580 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.580 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.580 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2024-04-21T10:41:04.580 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2024-04-21T10:41:04.580 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.581 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.581 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.581 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.581 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.581 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.581 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.581 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T10:41:04.581 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T10:41:04.582 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.582 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.582 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2024-04-21T10:41:04.582 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2024-04-21T10:41:04.582 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.582 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.583 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.583 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:04.583 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.583 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.583 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.583 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T10:41:04.583 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.584 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.584 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.584 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2024-04-21T10:41:04.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T10:41:04.584 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:04.584 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.585 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.585 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2024-04-21T10:41:04.585 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.585 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.585 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.585 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T10:41:04.585 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.585 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.586 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.586 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm_log_destination": { 2024-04-21T10:41:04.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm_log_destination", 2024-04-21T10:41:04.586 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.586 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.586 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.587 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.587 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.587 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.587 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.587 INFO:teuthology.orchestra.run.smithi026.stdout: "file", 2024-04-21T10:41:04.587 INFO:teuthology.orchestra.run.smithi026.stdout: "file,syslog", 2024-04-21T10:41:04.587 INFO:teuthology.orchestra.run.smithi026.stdout: "syslog" 2024-04-21T10:41:04.587 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.588 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T10:41:04.588 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.588 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.588 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.588 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2024-04-21T10:41:04.588 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2024-04-21T10:41:04.588 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.589 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.589 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.589 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.589 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.589 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.589 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.589 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T10:41:04.589 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.589 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.590 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.590 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2024-04-21T10:41:04.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2024-04-21T10:41:04.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.590 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.590 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.590 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.591 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.591 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.591 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.591 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T10:41:04.591 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.591 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.591 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.592 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2024-04-21T10:41:04.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2024-04-21T10:41:04.592 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.592 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.592 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.592 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.592 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.592 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.593 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.593 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T10:41:04.593 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.593 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.593 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.593 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2024-04-21T10:41:04.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2024-04-21T10:41:04.593 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.594 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.594 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.594 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T10:41:04.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.594 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.594 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.594 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-04-21T10:41:04.595 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.595 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.595 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.595 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2024-04-21T10:41:04.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2024-04-21T10:41:04.595 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.595 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.596 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.596 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T10:41:04.596 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.596 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.596 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.596 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2024-04-21T10:41:04.596 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.596 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.596 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.597 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2024-04-21T10:41:04.597 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2024-04-21T10:41:04.597 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.597 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.597 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.597 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T10:41:04.597 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.598 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.598 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.598 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2024-04-21T10:41:04.598 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.598 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.598 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.598 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2024-04-21T10:41:04.599 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2024-04-21T10:41:04.599 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.599 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.599 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.599 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T10:41:04.599 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.599 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.599 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.600 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-04-21T10:41:04.600 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.600 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.600 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.600 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2024-04-21T10:41:04.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2024-04-21T10:41:04.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.600 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.601 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.601 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T10:41:04.601 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.601 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.601 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.601 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2024-04-21T10:41:04.601 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.601 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.601 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.602 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2024-04-21T10:41:04.602 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2024-04-21T10:41:04.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.602 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.602 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.602 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T10:41:04.602 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.603 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.603 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.603 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2024-04-21T10:41:04.603 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.603 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.603 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.603 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2024-04-21T10:41:04.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2024-04-21T10:41:04.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.604 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.604 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.604 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T10:41:04.604 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.604 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.604 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.604 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2024-04-21T10:41:04.605 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.605 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.605 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.605 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2024-04-21T10:41:04.605 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2024-04-21T10:41:04.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.605 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.605 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.606 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T10:41:04.606 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.606 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.606 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.606 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2024-04-21T10:41:04.606 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.606 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.606 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.607 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2024-04-21T10:41:04.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2024-04-21T10:41:04.607 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.607 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.607 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.607 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T10:41:04.607 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.608 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.608 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.608 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2024-04-21T10:41:04.608 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.608 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.608 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.608 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2024-04-21T10:41:04.609 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2024-04-21T10:41:04.609 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.609 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.609 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.609 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T10:41:04.609 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.609 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.610 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.610 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2024-04-21T10:41:04.610 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.610 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.610 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.611 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2024-04-21T10:41:04.611 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2024-04-21T10:41:04.611 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.611 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.611 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.611 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T10:41:04.611 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.612 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.612 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.612 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-04-21T10:41:04.612 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.612 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.612 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.612 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_nvmeof": { 2024-04-21T10:41:04.613 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_nvmeof", 2024-04-21T10:41:04.613 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.613 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.613 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.613 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T10:41:04.613 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.613 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.614 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.614 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Nvme-of container image", 2024-04-21T10:41:04.614 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.614 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.614 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.614 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2024-04-21T10:41:04.615 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2024-04-21T10:41:04.615 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.615 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.615 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.615 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T10:41:04.615 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.615 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.615 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.616 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2024-04-21T10:41:04.616 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.616 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.616 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.616 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2024-04-21T10:41:04.616 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2024-04-21T10:41:04.617 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.617 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.617 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.617 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T10:41:04.617 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.617 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.617 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.617 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2024-04-21T10:41:04.618 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.618 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.618 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.618 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_samba": { 2024-04-21T10:41:04.618 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_samba", 2024-04-21T10:41:04.618 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.618 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.619 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.619 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T10:41:04.619 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.619 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.619 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.619 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Samba/SMB container image", 2024-04-21T10:41:04.620 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.620 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.620 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.620 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2024-04-21T10:41:04.620 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2024-04-21T10:41:04.620 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.620 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.621 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.621 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T10:41:04.621 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.621 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.621 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.621 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2024-04-21T10:41:04.621 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.622 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.622 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.622 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2024-04-21T10:41:04.622 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2024-04-21T10:41:04.622 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.622 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.623 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.623 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.623 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.623 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.623 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.623 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T10:41:04.623 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.624 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.624 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.624 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2024-04-21T10:41:04.624 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2024-04-21T10:41:04.624 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.624 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.625 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.625 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:04.625 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.625 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.625 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.625 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T10:41:04.625 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.625 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.626 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.626 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2024-04-21T10:41:04.626 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T10:41:04.626 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.626 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.626 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.627 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2024-04-21T10:41:04.627 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.627 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.627 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.627 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T10:41:04.627 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.627 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.627 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.628 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2024-04-21T10:41:04.628 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2024-04-21T10:41:04.628 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.628 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.628 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.628 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2024-04-21T10:41:04.628 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.628 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.629 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.629 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T10:41:04.629 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.629 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.629 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.629 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2024-04-21T10:41:04.629 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2024-04-21T10:41:04.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.630 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.630 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.630 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2024-04-21T10:41:04.630 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.630 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.630 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.630 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2024-04-21T10:41:04.631 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.631 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.631 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.631 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2024-04-21T10:41:04.631 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2024-04-21T10:41:04.631 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.631 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.631 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.632 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.632 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.632 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.632 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.632 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T10:41:04.632 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.632 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.633 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.633 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2024-04-21T10:41:04.633 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2024-04-21T10:41:04.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.633 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.634 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.634 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:04.634 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.634 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.634 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.634 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2024-04-21T10:41:04.634 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.634 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.635 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.635 INFO:teuthology.orchestra.run.smithi026.stdout: "grafana_dashboards_path": { 2024-04-21T10:41:04.635 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "grafana_dashboards_path", 2024-04-21T10:41:04.635 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.635 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.635 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.636 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T10:41:04.636 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.636 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.636 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.636 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T10:41:04.636 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.636 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.637 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.637 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2024-04-21T10:41:04.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2024-04-21T10:41:04.637 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.637 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.637 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.638 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:04.638 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.638 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.638 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.638 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2024-04-21T10:41:04.638 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.638 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.639 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.639 INFO:teuthology.orchestra.run.smithi026.stdout: "hw_monitoring": { 2024-04-21T10:41:04.639 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hw_monitoring", 2024-04-21T10:41:04.639 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.639 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.639 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.640 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.640 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.640 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.640 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.640 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T10:41:04.640 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.640 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.640 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.641 INFO:teuthology.orchestra.run.smithi026.stdout: "inventory_list_all": { 2024-04-21T10:41:04.641 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "inventory_list_all", 2024-04-21T10:41:04.641 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.641 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.641 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.641 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.641 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.642 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.642 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.642 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T10:41:04.642 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.642 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.642 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.643 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.643 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.643 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.643 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.643 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.643 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.643 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.644 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.644 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.644 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.644 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.644 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.644 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.644 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.644 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.645 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.645 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.645 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.645 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.645 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2024-04-21T10:41:04.645 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2024-04-21T10:41:04.646 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.646 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.646 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.646 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.646 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.646 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.646 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.647 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T10:41:04.647 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.647 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.647 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.647 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.647 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.647 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.648 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.648 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.648 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.648 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.648 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.648 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.648 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T10:41:04.648 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.648 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.649 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.649 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.649 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.649 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.649 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.649 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.649 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.650 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.650 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.650 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.650 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.650 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.650 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.650 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.650 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.651 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.651 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.651 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.651 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.651 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.651 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.651 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.651 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.652 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.652 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.652 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.652 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.652 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.652 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.652 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.652 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.652 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.653 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.653 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.653 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.653 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T10:41:04.653 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T10:41:04.653 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.653 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.653 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.654 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.654 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.654 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.654 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.654 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T10:41:04.654 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.654 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.654 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.655 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T10:41:04.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T10:41:04.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.655 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.655 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.655 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2024-04-21T10:41:04.655 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.655 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.656 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.656 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T10:41:04.656 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.656 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.656 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.656 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2024-04-21T10:41:04.656 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2024-04-21T10:41:04.656 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.657 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.657 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.657 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:04.657 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.657 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.657 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.657 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2024-04-21T10:41:04.657 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.658 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.658 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.658 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2024-04-21T10:41:04.658 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2024-04-21T10:41:04.658 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.658 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.658 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.659 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:04.659 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.659 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.659 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.659 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T10:41:04.659 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.659 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.659 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.660 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2024-04-21T10:41:04.660 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2024-04-21T10:41:04.660 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.660 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.660 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.660 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.660 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.660 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.661 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2024-04-21T10:41:04.661 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.661 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.661 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.661 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2024-04-21T10:41:04.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2024-04-21T10:41:04.661 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.662 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.662 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.662 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2024-04-21T10:41:04.662 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.662 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.662 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.662 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2024-04-21T10:41:04.662 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2024-04-21T10:41:04.663 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.663 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T10:41:04.663 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.663 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.663 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.663 INFO:teuthology.orchestra.run.smithi026.stdout: "oob_default_addr": { 2024-04-21T10:41:04.663 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "oob_default_addr", 2024-04-21T10:41:04.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.664 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.664 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.664 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "169.254.1.1", 2024-04-21T10:41:04.664 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.664 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.664 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.664 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T10:41:04.664 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.665 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.665 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.665 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2024-04-21T10:41:04.665 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2024-04-21T10:41:04.665 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.665 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.665 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.666 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T10:41:04.666 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.666 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.666 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.666 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T10:41:04.666 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.666 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.667 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.667 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2024-04-21T10:41:04.667 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2024-04-21T10:41:04.667 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.667 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.667 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.668 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.668 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.668 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.668 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.668 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T10:41:04.668 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.668 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.668 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.669 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2024-04-21T10:41:04.669 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2024-04-21T10:41:04.669 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.669 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.669 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.669 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.669 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.670 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.670 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T10:41:04.670 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.670 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.670 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.671 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2024-04-21T10:41:04.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2024-04-21T10:41:04.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.671 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.671 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.671 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.671 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.671 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.672 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.672 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T10:41:04.672 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.672 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.672 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.672 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2024-04-21T10:41:04.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2024-04-21T10:41:04.673 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.673 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.673 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.673 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.673 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.673 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.673 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.673 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T10:41:04.673 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.674 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.674 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.674 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2024-04-21T10:41:04.674 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2024-04-21T10:41:04.674 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.674 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.675 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.675 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.675 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.675 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.675 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.675 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T10:41:04.675 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.675 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.676 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.676 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2024-04-21T10:41:04.676 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2024-04-21T10:41:04.676 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.676 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.676 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.676 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2024-04-21T10:41:04.677 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.677 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.677 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.677 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2024-04-21T10:41:04.677 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.677 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.677 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.678 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2024-04-21T10:41:04.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2024-04-21T10:41:04.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.678 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.678 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.678 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.679 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.679 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.679 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.679 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T10:41:04.679 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.679 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.679 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.680 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2024-04-21T10:41:04.680 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2024-04-21T10:41:04.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.680 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.680 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.680 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.680 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.681 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.681 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.681 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T10:41:04.681 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.681 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.681 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.681 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2024-04-21T10:41:04.682 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2024-04-21T10:41:04.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.682 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.682 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.682 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.682 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.682 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.682 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.683 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T10:41:04.683 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.683 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.683 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.683 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2024-04-21T10:41:04.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2024-04-21T10:41:04.683 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.684 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.684 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.684 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.684 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.684 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.684 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.684 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T10:41:04.684 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.685 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.685 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.685 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2024-04-21T10:41:04.685 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2024-04-21T10:41:04.685 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.685 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.685 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.686 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.686 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.686 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.686 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.686 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T10:41:04.686 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.686 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.686 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.687 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2024-04-21T10:41:04.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2024-04-21T10:41:04.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.687 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.687 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.687 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.687 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.687 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.688 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.688 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T10:41:04.688 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.688 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.688 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.688 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.688 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2024-04-21T10:41:04.689 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.689 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.689 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.689 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.690 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.690 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.690 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.690 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.690 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.690 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.690 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.690 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.691 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.691 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.691 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.691 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.691 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.691 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.691 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.691 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.692 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.692 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.692 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.692 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.692 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.692 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.692 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.692 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.693 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.693 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.693 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.693 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.693 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.693 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.694 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.694 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.694 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.694 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.694 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.694 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.694 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.695 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.695 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.695 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.695 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.695 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.695 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.695 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.695 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.696 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.696 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.696 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.696 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.696 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.696 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.696 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.697 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.697 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.697 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.697 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.697 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.697 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.697 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.697 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.698 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.698 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.698 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.698 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2024-04-21T10:41:04.698 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2024-04-21T10:41:04.698 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.699 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.699 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.699 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2024-04-21T10:41:04.699 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.699 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.699 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.699 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T10:41:04.699 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.700 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.700 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.700 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2024-04-21T10:41:04.700 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2024-04-21T10:41:04.700 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.700 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.701 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.701 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2024-04-21T10:41:04.701 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.701 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.701 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.701 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T10:41:04.701 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.701 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.702 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.702 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.702 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.702 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.702 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2024-04-21T10:41:04.702 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.702 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.703 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.703 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T10:41:04.703 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T10:41:04.703 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.703 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.703 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.703 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:04.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.704 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.704 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.704 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.704 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.704 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.704 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.704 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T10:41:04.704 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T10:41:04.705 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.705 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.705 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.705 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.705 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.705 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.705 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.706 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.706 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.706 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.706 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.706 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T10:41:04.706 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T10:41:04.706 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.706 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.707 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.707 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.707 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.707 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.707 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.707 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.707 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.707 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.707 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.708 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2024-04-21T10:41:04.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T10:41:04.708 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.708 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.708 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.708 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.708 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.709 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.709 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.709 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.709 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.709 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.709 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.709 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.709 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T10:41:04.709 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T10:41:04.710 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.710 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.710 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.710 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.710 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.710 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.710 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.710 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.711 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.711 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.711 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.711 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T10:41:04.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T10:41:04.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.711 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.712 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.712 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.712 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.712 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.712 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.712 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.713 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T10:41:04.713 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T10:41:04.713 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.713 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.713 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.713 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.713 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.713 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.714 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.714 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.714 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.714 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.714 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T10:41:04.714 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T10:41:04.714 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.714 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.715 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.715 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.715 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.715 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.715 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.715 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.715 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.715 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.716 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T10:41:04.716 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T10:41:04.716 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.716 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.716 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.716 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.716 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.716 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.717 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.717 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.717 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.717 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.717 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.717 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.717 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T10:41:04.717 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T10:41:04.717 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.717 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.718 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.718 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.718 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.718 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.718 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.718 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.718 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.718 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.718 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.719 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T10:41:04.719 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T10:41:04.719 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.719 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.719 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.719 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.719 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.719 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.720 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.720 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.720 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.720 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.720 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.720 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T10:41:04.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T10:41:04.720 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.721 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.721 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.721 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.721 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.721 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.721 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.721 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.721 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.722 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.722 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.722 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T10:41:04.722 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T10:41:04.722 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.722 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.722 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.722 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.722 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.723 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.723 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.723 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.723 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.723 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.723 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.723 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T10:41:04.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T10:41:04.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.724 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.724 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.724 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.724 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.724 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.724 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.724 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.724 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.725 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.725 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.725 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T10:41:04.725 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T10:41:04.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.725 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.725 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.725 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-04-21T10:41:04.726 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.726 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.726 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.726 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.726 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.726 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.726 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.727 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T10:41:04.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T10:41:04.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.727 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.727 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.727 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.727 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.728 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.728 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.728 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.728 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.728 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.728 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.728 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2024-04-21T10:41:04.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2024-04-21T10:41:04.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.729 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.729 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.729 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.729 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.729 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.729 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.730 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.730 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.730 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.730 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.730 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T10:41:04.730 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T10:41:04.731 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.731 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.731 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.731 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-04-21T10:41:04.731 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.731 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.731 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.731 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.732 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.732 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.732 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.732 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T10:41:04.732 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T10:41:04.733 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.733 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.733 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.733 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.733 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.733 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.733 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.733 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.733 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.734 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.734 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.734 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T10:41:04.734 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T10:41:04.734 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.734 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.734 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.735 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.735 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.735 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.735 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.735 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.735 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.735 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.736 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.736 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T10:41:04.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T10:41:04.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.736 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.736 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.736 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.736 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.737 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.737 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.737 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.737 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.737 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.737 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.737 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T10:41:04.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T10:41:04.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.738 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.738 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.738 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.738 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.738 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.739 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.739 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.739 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.739 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.739 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.739 INFO:teuthology.orchestra.run.smithi026.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T10:41:04.739 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T10:41:04.740 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.740 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.740 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.740 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "[]", 2024-04-21T10:41:04.740 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.740 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.740 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.740 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.741 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.741 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.741 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.741 INFO:teuthology.orchestra.run.smithi026.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T10:41:04.741 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T10:41:04.741 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.741 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.742 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.742 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "{}", 2024-04-21T10:41:04.742 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.742 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.742 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.742 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.742 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.743 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.743 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.743 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T10:41:04.743 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T10:41:04.743 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.743 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.743 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.744 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.744 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.744 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.744 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.744 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.744 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.744 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.744 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.745 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T10:41:04.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T10:41:04.745 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.745 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.745 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.745 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.745 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.746 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.746 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.746 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.746 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.746 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.746 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.746 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T10:41:04.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T10:41:04.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.747 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.747 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.747 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.747 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.747 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.747 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.748 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.748 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.748 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.748 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.748 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T10:41:04.748 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T10:41:04.748 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.748 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.749 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.749 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.749 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.749 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.749 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.749 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.749 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.749 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.749 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.750 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T10:41:04.750 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T10:41:04.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.750 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.750 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.750 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.751 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.751 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.751 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.751 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.751 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.751 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.751 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.752 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T10:41:04.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T10:41:04.752 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.752 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.752 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.752 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.752 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.752 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.752 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.753 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.753 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.753 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.753 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.753 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T10:41:04.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T10:41:04.753 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.754 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.754 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.754 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.754 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.754 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.754 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.754 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.754 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.755 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.755 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.755 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T10:41:04.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T10:41:04.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.755 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.755 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.755 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.756 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.756 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.756 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.756 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.756 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.756 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.756 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.757 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T10:41:04.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T10:41:04.757 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.757 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.757 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.757 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.757 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.757 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.757 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.758 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.758 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.758 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.758 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.758 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T10:41:04.758 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T10:41:04.758 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.759 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.759 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.759 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.759 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.759 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.759 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.759 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.760 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.760 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.760 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.760 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T10:41:04.760 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T10:41:04.760 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.761 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.761 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.761 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T10:41:04.761 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.761 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.761 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.761 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.761 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.762 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.762 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.762 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T10:41:04.762 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T10:41:04.762 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.763 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.763 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.763 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:04.763 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.763 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.763 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.763 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.763 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.764 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.764 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.764 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T10:41:04.764 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T10:41:04.764 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.764 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.764 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.765 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2024-04-21T10:41:04.765 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.765 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.765 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.765 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.765 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.765 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.766 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.766 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T10:41:04.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T10:41:04.766 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.766 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.766 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.766 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2024-04-21T10:41:04.767 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.767 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.767 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.767 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.767 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.767 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.767 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.768 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T10:41:04.768 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T10:41:04.768 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.768 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.768 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.768 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.768 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.768 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.768 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.769 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.769 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.769 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.769 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.769 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T10:41:04.769 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T10:41:04.769 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.770 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.770 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.770 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2024-04-21T10:41:04.770 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.770 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.770 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.770 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.771 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.771 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.771 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.771 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T10:41:04.771 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T10:41:04.771 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.771 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.771 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.773 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.773 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.773 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.773 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.773 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.773 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.774 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.774 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.774 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T10:41:04.774 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T10:41:04.774 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.774 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.774 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.775 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.775 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.775 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.775 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.775 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.775 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.775 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.775 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.776 INFO:teuthology.orchestra.run.smithi026.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T10:41:04.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T10:41:04.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.776 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.776 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.776 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.776 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.776 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.776 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.776 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.777 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.777 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.777 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.777 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T10:41:04.777 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T10:41:04.777 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.777 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.777 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.778 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:04.778 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.778 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.778 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.778 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.778 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.778 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.778 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.779 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T10:41:04.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T10:41:04.779 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.779 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.779 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.779 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:04.779 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.780 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.780 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.780 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.780 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.780 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.780 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.780 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T10:41:04.780 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T10:41:04.781 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.781 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.781 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.781 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-04-21T10:41:04.781 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.781 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.781 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.781 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.781 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.782 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.782 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.782 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2024-04-21T10:41:04.782 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2024-04-21T10:41:04.782 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.782 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.782 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.783 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.783 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.783 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.783 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.783 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.783 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.783 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.783 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.784 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2024-04-21T10:41:04.784 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2024-04-21T10:41:04.784 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.784 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.784 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.784 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.784 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.784 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.784 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.785 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.785 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.785 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.785 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.785 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2024-04-21T10:41:04.785 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2024-04-21T10:41:04.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.786 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.786 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.786 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.786 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.786 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.786 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.786 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2024-04-21T10:41:04.786 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.787 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.787 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.787 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2024-04-21T10:41:04.787 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2024-04-21T10:41:04.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.787 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.788 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.788 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2024-04-21T10:41:04.788 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.788 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.788 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.788 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.788 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.788 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.789 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.789 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-04-21T10:41:04.789 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-04-21T10:41:04.789 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.789 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.789 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.789 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.790 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.790 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.790 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.790 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.790 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.790 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.790 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.791 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.791 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.791 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.791 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.791 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.791 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.791 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.792 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.792 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.792 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.792 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.792 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.792 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.792 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.793 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.793 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.793 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.793 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.793 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.793 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.793 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.793 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.794 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.794 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.794 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.794 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.794 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.794 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.794 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.795 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.795 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.795 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.795 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.795 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.795 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.795 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.795 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.796 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.796 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.796 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.796 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.796 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.796 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.796 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.796 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.797 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.797 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.797 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.797 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.797 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.797 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.797 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.798 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.798 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.798 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.798 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.798 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.798 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.798 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.799 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.799 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.799 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.799 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.799 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.799 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.799 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2024-04-21T10:41:04.800 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2024-04-21T10:41:04.800 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.800 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.800 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.800 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.800 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.800 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.801 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.801 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2024-04-21T10:41:04.801 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.801 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.801 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.801 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2024-04-21T10:41:04.801 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T10:41:04.802 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.802 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.802 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.802 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.802 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.802 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.802 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.803 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.803 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.803 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.803 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.803 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-04-21T10:41:04.803 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-04-21T10:41:04.803 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.803 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.804 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.804 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-04-21T10:41:04.804 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.804 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.804 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.804 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.804 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.804 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.804 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.805 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-04-21T10:41:04.805 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-04-21T10:41:04.805 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.805 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.805 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.805 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2024-04-21T10:41:04.805 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.806 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.806 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.806 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.806 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.806 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.806 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.807 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-04-21T10:41:04.807 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-04-21T10:41:04.807 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.807 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.807 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.807 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.807 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.807 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.808 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.808 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.808 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.808 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.808 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.808 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2024-04-21T10:41:04.808 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2024-04-21T10:41:04.809 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.809 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.809 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.809 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2024-04-21T10:41:04.809 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.809 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.809 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.809 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.810 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.810 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.810 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.810 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-04-21T10:41:04.810 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-04-21T10:41:04.810 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.811 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.811 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.811 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2024-04-21T10:41:04.811 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.811 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.811 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.811 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.811 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2024-04-21T10:41:04.812 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.812 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.812 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.812 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.812 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.812 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-04-21T10:41:04.813 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-04-21T10:41:04.813 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.813 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.813 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.813 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-04-21T10:41:04.813 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-04-21T10:41:04.813 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-04-21T10:41:04.813 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.814 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.814 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.814 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.814 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.814 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2024-04-21T10:41:04.814 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2024-04-21T10:41:04.815 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.815 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.815 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.815 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.815 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.815 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.815 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.816 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.816 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.816 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.816 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.816 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.816 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.817 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-04-21T10:41:04.817 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.817 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.817 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.817 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2024-04-21T10:41:04.817 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2024-04-21T10:41:04.817 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.817 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.818 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.818 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.818 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.818 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.818 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.818 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2024-04-21T10:41:04.818 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.818 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.819 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.819 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.819 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.819 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.819 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.819 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.819 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.820 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.820 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.820 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.820 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.820 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.820 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.820 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.820 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.821 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.821 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.821 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.821 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.821 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.821 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.822 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.822 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.822 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.822 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.822 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.822 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.823 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.823 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.823 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.823 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.823 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.823 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.823 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.824 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.824 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.824 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.824 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.824 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.824 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.824 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.825 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.825 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.825 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.825 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.825 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.825 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.825 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.826 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.826 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.826 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.826 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.826 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.826 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.827 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.827 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.827 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.827 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.827 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.827 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.827 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.827 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.828 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.828 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.828 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.828 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2024-04-21T10:41:04.828 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2024-04-21T10:41:04.828 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.828 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.829 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.829 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2024-04-21T10:41:04.829 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.829 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.829 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.829 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T10:41:04.829 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.829 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.830 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.830 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2024-04-21T10:41:04.830 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2024-04-21T10:41:04.830 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.830 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.830 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.831 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2024-04-21T10:41:04.831 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.831 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.831 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.831 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T10:41:04.831 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.831 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.831 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.832 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2024-04-21T10:41:04.832 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2024-04-21T10:41:04.832 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.832 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.832 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.832 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2024-04-21T10:41:04.832 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.833 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.833 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.833 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2024-04-21T10:41:04.833 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.833 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.833 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.834 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2024-04-21T10:41:04.834 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2024-04-21T10:41:04.834 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.834 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.834 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.834 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-04-21T10:41:04.834 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.835 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.835 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.835 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T10:41:04.835 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.835 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.835 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.835 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2024-04-21T10:41:04.835 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2024-04-21T10:41:04.836 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.836 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.836 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.836 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:04.836 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.836 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.836 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.836 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T10:41:04.836 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.837 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.837 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.837 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:04.837 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:04.837 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.837 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.837 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.837 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:04.838 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.838 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.838 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.838 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T10:41:04.838 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.838 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.838 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.839 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2024-04-21T10:41:04.839 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2024-04-21T10:41:04.839 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.839 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.839 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.839 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2024-04-21T10:41:04.839 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.839 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.840 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.840 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T10:41:04.840 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.840 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.840 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.840 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.840 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.841 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2024-04-21T10:41:04.841 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.841 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.841 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.841 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.841 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.841 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.842 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.842 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.842 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.842 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.842 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.842 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.842 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.842 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.843 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.843 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.843 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.843 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.843 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.843 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.843 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.843 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.844 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.844 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.844 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.844 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.844 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.845 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.845 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.845 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.845 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.845 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.845 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.845 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.846 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.846 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.846 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.846 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.846 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.846 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.847 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.847 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.847 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.847 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.847 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.847 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.847 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.848 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.848 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.848 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.848 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.848 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.848 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.848 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.849 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.849 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.849 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.849 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.849 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.850 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.850 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.850 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.850 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.850 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.850 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.850 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.851 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.851 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2024-04-21T10:41:04.851 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2024-04-21T10:41:04.851 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.851 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.851 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.851 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2024-04-21T10:41:04.852 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.852 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.852 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.852 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.852 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.852 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.852 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.853 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2024-04-21T10:41:04.853 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2024-04-21T10:41:04.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.853 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.853 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.853 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2024-04-21T10:41:04.853 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.853 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.854 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.854 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.854 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.854 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.854 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.854 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:04.855 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:04.855 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.855 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.855 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.855 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2024-04-21T10:41:04.855 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.855 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.855 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.856 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.856 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.856 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.856 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.856 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.856 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2024-04-21T10:41:04.857 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2024-04-21T10:41:04.857 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2024-04-21T10:41:04.857 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.857 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2024-04-21T10:41:04.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2024-04-21T10:41:04.857 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.857 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.858 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.858 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2024-04-21T10:41:04.858 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.858 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.858 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.858 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T10:41:04.858 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.858 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.859 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.859 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2024-04-21T10:41:04.859 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2024-04-21T10:41:04.859 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.859 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.859 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.859 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2024-04-21T10:41:04.860 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.860 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.860 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.860 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T10:41:04.860 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.860 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.860 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.861 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2024-04-21T10:41:04.861 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2024-04-21T10:41:04.861 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.861 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.861 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.861 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.861 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.861 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.861 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.862 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2024-04-21T10:41:04.862 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.862 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.862 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.862 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:04.862 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:04.862 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:04.862 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.863 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.863 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2024-04-21T10:41:04.863 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2024-04-21T10:41:04.863 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.863 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.863 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T10:41:04.863 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.863 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.863 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.864 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.864 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.864 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.864 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.864 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.864 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.864 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.865 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.865 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.865 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.865 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.865 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.865 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.865 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.865 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.866 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.866 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.866 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.866 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.866 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.866 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.866 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.867 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.867 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.867 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.867 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.867 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.867 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.867 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.867 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.867 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.868 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.868 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.868 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.868 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.868 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.868 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.868 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.868 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.869 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.869 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.869 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.869 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.869 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.869 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.869 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.869 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.869 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.870 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.870 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.870 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.870 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.870 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.870 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.870 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.871 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.871 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.871 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.871 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.871 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.871 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.871 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.871 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.871 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.871 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.872 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.872 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2024-04-21T10:41:04.872 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2024-04-21T10:41:04.872 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.872 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.872 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.872 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.872 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.873 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.873 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.873 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2024-04-21T10:41:04.873 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.873 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.873 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.873 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2024-04-21T10:41:04.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2024-04-21T10:41:04.874 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.874 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.874 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.874 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2024-04-21T10:41:04.874 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.874 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.874 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.874 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2024-04-21T10:41:04.875 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.875 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.875 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.875 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2024-04-21T10:41:04.875 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2024-04-21T10:41:04.875 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.875 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.875 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.876 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2024-04-21T10:41:04.876 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.876 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.876 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.876 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T10:41:04.876 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.876 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.876 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.877 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2024-04-21T10:41:04.877 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2024-04-21T10:41:04.877 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.877 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.877 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.877 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-04-21T10:41:04.877 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-04-21T10:41:04.877 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2024-04-21T10:41:04.878 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.878 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T10:41:04.878 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.878 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.878 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.878 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2024-04-21T10:41:04.878 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2024-04-21T10:41:04.878 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.879 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.879 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.879 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.879 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.879 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.879 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.879 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2024-04-21T10:41:04.879 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.879 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.880 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.880 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2024-04-21T10:41:04.880 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2024-04-21T10:41:04.880 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.880 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.880 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.880 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2024-04-21T10:41:04.880 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.880 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.881 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.881 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T10:41:04.881 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.881 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.881 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.881 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.881 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2024-04-21T10:41:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.885 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.885 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.885 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.885 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.885 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.885 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.885 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.885 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.885 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.886 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.886 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.886 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.886 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.886 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.886 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.887 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.887 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.887 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.887 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.887 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.887 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.887 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.887 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.887 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.888 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.888 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.888 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.888 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.888 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.888 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.888 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.888 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.888 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.889 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.889 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.889 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.889 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.889 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.889 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.889 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.889 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.889 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.890 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.890 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.890 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.890 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.890 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.890 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.890 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2024-04-21T10:41:04.891 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.891 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.891 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.891 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.891 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.891 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.891 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.891 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.891 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.892 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.892 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.892 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.892 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.892 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.892 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.892 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.892 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.892 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.893 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.893 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.893 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.893 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.893 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.893 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.893 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.894 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.894 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.894 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.894 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.894 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.894 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.894 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.894 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.894 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.894 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.895 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.895 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.895 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.895 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.895 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.895 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.895 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.896 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.896 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.896 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.896 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.896 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.896 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.896 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.896 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.896 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.897 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.897 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.897 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.897 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.897 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.897 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.897 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.897 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.898 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2024-04-21T10:41:04.899 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.899 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.899 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.899 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2024-04-21T10:41:04.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2024-04-21T10:41:04.899 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.900 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.900 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.900 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2024-04-21T10:41:04.900 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.900 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.900 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.900 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T10:41:04.900 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.901 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.901 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.901 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2024-04-21T10:41:04.901 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2024-04-21T10:41:04.901 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.901 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.901 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.902 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:04.902 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2024-04-21T10:41:04.902 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.902 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.902 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T10:41:04.902 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.902 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.902 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.903 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.903 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.903 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.903 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.903 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.903 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.904 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.904 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.904 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.904 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.904 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.904 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.904 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.904 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.904 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.905 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.905 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.905 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.905 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.905 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.905 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.906 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.906 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.906 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.906 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.906 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.906 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.906 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.906 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.906 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.907 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.907 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.907 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.907 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.907 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.907 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.907 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.908 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.908 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.908 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.908 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.908 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.908 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.908 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.908 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.909 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.909 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.909 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.909 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.909 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.909 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.909 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.909 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.909 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.910 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.910 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.910 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.910 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.910 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.910 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.910 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.910 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.911 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.911 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.911 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.911 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.911 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.911 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2024-04-21T10:41:04.911 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.911 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.912 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.912 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2024-04-21T10:41:04.912 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2024-04-21T10:41:04.912 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.912 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.912 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.912 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2024-04-21T10:41:04.912 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.913 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.913 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.913 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2024-04-21T10:41:04.913 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T10:41:04.913 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.913 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.913 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.914 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.914 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.914 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.914 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.914 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.914 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.914 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.915 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.915 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.915 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.915 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.915 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.915 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.915 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.915 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.916 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.916 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.916 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.916 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.916 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.916 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.917 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.917 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.917 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.917 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.917 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.917 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.917 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.918 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.918 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.918 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.918 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.918 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.918 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.919 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.919 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.919 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.919 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.919 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.919 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.919 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.919 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.920 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.920 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.920 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.920 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.920 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.920 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.920 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.921 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.921 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.921 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.921 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.921 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.921 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.922 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.922 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.922 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.922 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.922 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.922 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.922 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.923 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.923 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.923 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.923 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2024-04-21T10:41:04.923 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2024-04-21T10:41:04.923 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.923 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.924 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.924 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.924 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.924 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.924 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.924 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2024-04-21T10:41:04.924 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T10:41:04.924 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.925 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.925 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2024-04-21T10:41:04.925 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2024-04-21T10:41:04.925 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.925 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.925 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.925 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2024-04-21T10:41:04.926 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.926 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.926 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.926 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2024-04-21T10:41:04.926 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.926 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.926 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.926 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2024-04-21T10:41:04.927 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2024-04-21T10:41:04.927 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:04.927 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.927 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.927 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2024-04-21T10:41:04.927 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.927 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.927 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.928 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T10:41:04.928 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.928 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.928 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.928 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2024-04-21T10:41:04.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2024-04-21T10:41:04.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.929 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.929 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.929 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.929 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.929 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.929 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.929 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2024-04-21T10:41:04.930 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.930 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.930 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.930 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2024-04-21T10:41:04.930 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2024-04-21T10:41:04.930 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.931 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.931 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.931 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2024-04-21T10:41:04.931 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.931 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.931 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.931 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T10:41:04.931 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T10:41:04.932 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.932 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.932 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.932 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.932 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.933 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2024-04-21T10:41:04.933 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.933 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.933 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.933 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.933 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.933 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.934 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.934 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.934 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.934 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.934 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.934 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.935 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.935 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.935 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.935 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.935 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.935 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.935 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.936 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.936 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.936 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.936 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.936 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.937 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.937 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.937 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.937 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.937 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.937 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.937 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.938 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.938 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.938 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.938 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.938 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.938 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.939 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.939 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.939 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.939 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.939 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.939 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.939 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.939 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.940 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.940 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.940 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.940 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.940 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.940 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.940 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.941 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.941 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.941 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.941 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.941 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.941 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.942 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.942 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.942 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.942 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.942 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.942 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.942 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.942 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.943 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.943 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.943 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.943 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.943 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.943 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.943 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2024-04-21T10:41:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.944 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.945 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.947 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.948 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.949 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.950 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.952 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2024-04-21T10:41:04.953 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.954 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.955 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.956 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.957 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.958 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.961 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-04-21T10:41:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.963 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2024-04-21T10:41:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2024-04-21T10:41:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.964 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T10:41:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.965 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.966 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.967 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.968 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.969 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.970 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.971 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.972 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.973 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2024-04-21T10:41:04.974 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2024-04-21T10:41:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.975 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2024-04-21T10:41:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2024-04-21T10:41:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2024-04-21T10:41:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2024-04-21T10:41:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.977 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2024-04-21T10:41:04.978 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.979 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.980 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.982 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.983 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.984 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.985 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.986 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.986 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.986 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.986 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.986 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.986 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.987 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.987 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.987 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.987 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.987 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.987 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.987 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.987 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2024-04-21T10:41:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.989 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.990 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.991 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.992 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.993 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.994 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:04.995 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:04.996 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:04.997 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:04.998 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2024-04-21T10:41:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:04.999 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.000 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.001 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.002 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.003 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.004 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.005 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.006 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.007 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.008 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2024-04-21T10:41:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2024-04-21T10:41:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2024-04-21T10:41:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2024-04-21T10:41:05.010 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2024-04-21T10:41:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T10:41:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2024-04-21T10:41:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2024-04-21T10:41:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2024-04-21T10:41:05.012 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T10:41:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.013 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-04-21T10:41:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-04-21T10:41:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.014 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.016 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.017 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.018 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.019 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.020 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.021 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.022 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2024-04-21T10:41:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2024-04-21T10:41:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.024 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2024-04-21T10:41:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2024-04-21T10:41:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2024-04-21T10:41:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2024-04-21T10:41:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2024-04-21T10:41:05.026 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2024-04-21T10:41:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2024-04-21T10:41:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2024-04-21T10:41:05.028 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2024-04-21T10:41:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.029 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2024-04-21T10:41:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2024-04-21T10:41:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.030 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T10:41:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T10:41:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.034 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.035 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.036 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.037 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.039 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2024-04-21T10:41:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2024-04-21T10:41:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.040 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.041 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T10:41:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T10:41:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:05.042 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2024-04-21T10:41:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.043 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2024-04-21T10:41:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2024-04-21T10:41:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.044 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2024-04-21T10:41:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.045 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-04-21T10:41:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-04-21T10:41:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2024-04-21T10:41:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.046 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T10:41:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-04-21T10:41:05.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-04-21T10:41:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2024-04-21T10:41:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.048 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T10:41:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2024-04-21T10:41:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2024-04-21T10:41:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.049 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2024-04-21T10:41:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.050 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.051 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.051 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.051 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2024-04-21T10:41:05.051 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2024-04-21T10:41:05.051 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.051 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.051 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.051 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2024-04-21T10:41:05.052 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.052 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.052 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.052 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2024-04-21T10:41:05.052 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2024-04-21T10:41:05.052 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.052 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.053 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.053 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.053 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.053 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2024-04-21T10:41:05.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2024-04-21T10:41:05.053 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:05.053 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.054 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.054 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2024-04-21T10:41:05.054 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2024-04-21T10:41:05.054 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2024-04-21T10:41:05.054 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.054 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.054 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.054 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.055 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.055 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.055 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-04-21T10:41:05.055 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.055 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.056 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.056 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.056 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.056 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.056 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.056 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.056 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.056 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.056 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.057 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.057 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.057 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.057 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.057 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.057 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.057 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.057 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.057 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.058 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.058 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.058 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.058 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.058 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.058 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.058 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.059 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.059 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.059 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.059 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.059 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.059 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.059 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.059 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.059 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.060 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.060 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.060 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.060 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.060 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.060 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.060 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.061 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.061 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.061 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.061 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.061 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.061 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.061 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.061 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.061 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.062 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.062 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.062 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.062 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.062 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.062 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.063 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.063 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.063 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.063 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.063 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.063 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.063 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.063 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.064 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.064 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.064 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.064 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2024-04-21T10:41:05.064 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2024-04-21T10:41:05.064 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:05.064 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.065 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.065 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2024-04-21T10:41:05.065 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.065 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.065 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.065 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.065 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.066 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.066 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.066 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2024-04-21T10:41:05.066 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T10:41:05.066 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.066 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.066 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.067 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.067 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.067 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.067 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.067 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.067 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.067 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.068 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.068 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2024-04-21T10:41:05.068 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2024-04-21T10:41:05.068 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.068 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.068 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.068 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.069 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.069 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.069 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.069 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.069 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.069 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.069 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.069 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.070 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.070 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.070 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2024-04-21T10:41:05.070 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.070 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.070 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.071 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2024-04-21T10:41:05.071 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2024-04-21T10:41:05.071 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.071 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.071 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.071 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:05.071 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.071 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.072 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.072 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.072 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.072 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.072 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.072 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2024-04-21T10:41:05.072 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2024-04-21T10:41:05.073 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.073 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.073 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.073 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.073 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.073 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.073 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.073 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.074 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.074 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.074 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.074 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.074 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.074 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.075 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.075 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.075 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.075 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.075 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.075 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.075 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.075 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.076 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.076 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.076 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.076 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.076 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.076 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.076 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.076 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.076 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.077 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.077 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.077 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.077 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.077 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.078 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.078 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.078 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.078 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.078 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.078 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.078 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.079 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.079 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.079 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.079 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.079 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.079 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.079 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.079 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.079 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.080 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.080 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.080 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.080 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.080 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.080 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.080 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.080 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.080 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.081 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.081 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.081 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.081 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.081 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.081 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.081 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.081 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.082 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.082 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.082 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.082 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.082 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.082 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.082 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2024-04-21T10:41:05.082 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2024-04-21T10:41:05.083 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.083 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.083 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.083 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.083 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.083 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.083 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.083 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.083 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.084 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.084 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.084 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2024-04-21T10:41:05.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2024-04-21T10:41:05.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.084 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.084 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.084 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.085 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.085 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.085 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.085 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.085 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.085 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.085 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.085 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.085 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.086 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2024-04-21T10:41:05.086 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.086 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.086 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.086 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.086 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.087 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.087 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.087 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.087 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.087 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.087 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.087 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.087 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.087 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.087 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.088 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.088 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.088 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.088 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.088 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.088 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.088 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.088 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.089 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.089 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.089 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.089 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.089 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.089 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.089 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.089 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.090 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.090 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.090 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.090 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.090 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.090 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.090 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.091 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.091 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.091 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.091 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.091 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.091 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.091 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.091 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.091 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.092 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.092 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.092 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.092 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.092 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.092 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.092 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.092 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.092 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.093 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.093 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.093 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.093 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.093 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.093 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.093 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.093 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.094 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.094 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.094 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.094 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.094 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.094 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.094 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.095 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.095 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2024-04-21T10:41:05.095 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.095 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.095 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.095 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.095 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.095 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.095 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.096 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.096 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.096 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.096 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.096 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.096 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.096 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.096 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.097 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.097 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.097 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.097 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.097 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.097 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.097 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.097 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.098 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.098 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.098 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.098 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.098 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.098 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.098 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.098 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.099 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.099 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.099 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.099 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.099 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.099 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.099 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.100 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.100 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.100 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.100 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.100 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.100 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.100 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.100 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.100 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.101 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.101 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.101 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.101 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.101 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.101 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.101 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.101 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.102 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.102 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.102 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.102 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.102 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.102 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.102 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.103 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.103 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.103 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.103 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.103 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.103 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.103 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.103 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2024-04-21T10:41:05.104 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2024-04-21T10:41:05.104 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.104 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.104 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.104 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2024-04-21T10:41:05.104 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.104 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.104 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.105 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T10:41:05.105 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.105 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.105 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.105 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.105 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.105 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.105 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2024-04-21T10:41:05.106 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.106 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.106 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.106 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.106 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.106 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.106 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.107 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.107 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.107 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.107 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.107 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.107 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.107 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.108 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.108 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.108 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.108 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.108 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.108 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.108 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.108 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.108 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.109 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.109 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.109 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.109 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.109 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.109 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.110 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.110 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.110 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.110 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.110 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.110 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.110 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.111 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.111 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.111 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.111 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.111 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.111 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.112 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.112 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.112 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.112 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.112 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.112 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.112 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.112 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.113 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.113 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.113 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.113 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.113 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.113 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.113 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.114 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.114 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.114 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.114 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.114 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.114 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.114 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.114 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.115 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.115 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.115 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2024-04-21T10:41:05.115 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2024-04-21T10:41:05.115 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.115 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.115 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.116 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.116 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.116 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.116 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.116 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.116 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.116 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.116 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.117 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2024-04-21T10:41:05.117 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2024-04-21T10:41:05.117 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.117 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.117 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.117 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2024-04-21T10:41:05.118 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.118 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.118 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.118 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.118 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.118 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.118 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.119 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2024-04-21T10:41:05.119 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2024-04-21T10:41:05.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.119 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.119 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.119 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.119 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.120 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.120 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.120 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.120 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.120 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.120 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.120 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2024-04-21T10:41:05.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2024-04-21T10:41:05.121 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:05.121 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.121 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.121 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.121 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.121 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.121 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.122 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.122 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.122 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.122 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.122 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2024-04-21T10:41:05.122 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2024-04-21T10:41:05.122 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2024-04-21T10:41:05.123 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.123 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.123 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.123 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.123 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.123 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.123 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.124 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.124 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.124 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.124 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.124 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2024-04-21T10:41:05.124 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2024-04-21T10:41:05.124 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.124 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.125 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.125 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.125 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.125 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.125 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.125 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.125 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.125 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.126 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.126 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2024-04-21T10:41:05.126 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2024-04-21T10:41:05.126 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.126 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.126 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.126 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2024-04-21T10:41:05.128 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2024-04-21T10:41:05.128 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.128 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.128 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.128 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:05.128 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.128 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.129 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.129 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.129 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.129 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.129 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.129 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2024-04-21T10:41:05.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2024-04-21T10:41:05.130 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:05.130 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.130 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.130 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.130 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2024-04-21T10:41:05.130 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2024-04-21T10:41:05.130 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.131 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.131 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.131 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.131 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.131 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2024-04-21T10:41:05.131 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2024-04-21T10:41:05.131 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.132 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.132 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.132 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.132 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.132 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.132 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.132 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.133 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.133 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.133 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.133 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2024-04-21T10:41:05.133 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2024-04-21T10:41:05.133 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.133 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.134 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.134 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.134 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.134 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.134 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.134 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.134 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.134 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.135 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.135 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2024-04-21T10:41:05.135 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2024-04-21T10:41:05.135 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.135 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.135 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.136 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.136 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.136 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.136 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.136 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.136 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.136 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.136 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.137 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.137 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-04-21T10:41:05.137 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.137 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.138 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.138 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2024-04-21T10:41:05.138 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2024-04-21T10:41:05.138 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.138 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.138 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.138 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.138 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.139 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.139 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.139 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T10:41:05.139 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.139 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.139 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.139 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2024-04-21T10:41:05.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2024-04-21T10:41:05.140 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.140 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.140 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.140 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.140 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.140 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.140 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.141 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2024-04-21T10:41:05.141 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.141 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.141 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.141 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.141 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.141 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.142 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.142 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.142 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.142 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.142 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.142 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.142 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.143 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.143 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.143 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.143 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.143 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.143 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.143 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.143 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.144 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.144 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.144 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.144 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.144 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.144 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.144 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.145 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.145 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.145 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.145 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.145 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.145 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.145 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.145 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.146 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.146 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.146 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.146 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.146 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.146 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.146 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.146 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.147 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.147 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.147 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.147 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.147 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.147 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.147 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.147 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.147 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.148 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.148 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.148 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.148 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.148 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.148 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.148 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.149 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.149 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.149 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.149 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.149 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.149 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.149 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.149 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.149 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.150 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.150 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2024-04-21T10:41:05.150 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.150 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.151 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.151 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.151 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.151 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.151 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.151 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.151 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.151 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.152 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.152 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.152 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.152 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.152 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.152 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.152 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.152 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.153 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.153 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.153 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.153 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.153 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.153 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.153 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.153 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.154 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.154 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.154 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.154 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.154 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.154 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.154 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.155 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.155 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.155 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.155 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.155 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.155 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.155 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.155 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.156 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.156 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.156 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.156 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.156 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.156 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.156 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.157 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.157 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.157 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.157 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.157 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.157 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.157 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.158 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.158 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.158 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.158 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.158 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.159 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.159 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.159 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.159 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.159 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.159 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.159 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.160 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.160 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.160 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.160 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.160 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.160 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.160 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2024-04-21T10:41:05.160 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.161 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.161 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.161 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.161 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.161 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.161 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.161 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.162 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.162 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.162 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.162 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.162 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.162 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.162 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.163 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.163 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.163 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.163 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.163 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.163 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.164 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.164 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.164 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.164 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.164 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.164 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.164 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.164 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.165 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.165 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.165 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.165 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.165 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.165 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.165 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.165 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.166 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.166 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.166 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.166 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.166 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.166 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.166 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.166 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.167 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.167 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.167 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.167 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.167 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.167 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.167 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.168 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.168 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.168 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.168 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.168 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.168 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.168 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.169 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.169 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.169 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.169 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.169 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.169 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.169 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.169 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.170 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.170 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.170 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2024-04-21T10:41:05.171 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.171 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.171 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.171 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2024-04-21T10:41:05.171 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2024-04-21T10:41:05.171 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.171 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.171 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.172 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T10:41:05.172 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.172 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.172 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.172 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.172 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.172 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.172 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.173 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:05.173 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:05.173 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:05.173 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.173 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.173 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2024-04-21T10:41:05.173 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.174 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.174 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.174 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.174 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.174 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.174 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.174 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.174 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.175 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.175 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.175 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.175 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.175 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.175 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.175 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.175 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.175 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.176 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.176 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.176 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.176 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.176 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.176 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.176 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.176 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.176 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.177 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.177 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.177 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.177 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.177 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.177 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.177 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.177 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.177 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.178 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.178 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.178 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.178 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.178 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.178 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.178 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.179 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.179 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.179 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.179 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.179 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.179 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.179 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.179 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.179 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.181 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.181 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.181 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.181 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.181 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.181 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.182 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.182 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.182 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.182 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.182 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.182 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.182 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.182 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.183 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.183 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.183 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.183 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.183 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.183 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.183 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.183 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.183 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.184 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.184 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.184 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.184 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2024-04-21T10:41:05.184 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.184 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.184 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.184 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2024-04-21T10:41:05.185 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2024-04-21T10:41:05.185 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.185 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.185 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.185 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:05.185 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.185 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.185 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.185 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T10:41:05.186 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.186 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.186 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.186 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2024-04-21T10:41:05.186 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2024-04-21T10:41:05.186 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.187 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.187 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.187 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:05.187 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.187 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.187 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.187 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T10:41:05.187 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.188 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.188 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.188 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.188 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2024-04-21T10:41:05.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2024-04-21T10:41:05.188 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.188 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.188 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.189 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:05.189 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.189 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.189 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.189 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T10:41:05.189 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.189 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.189 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.190 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2024-04-21T10:41:05.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2024-04-21T10:41:05.190 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.190 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.190 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.190 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.190 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.191 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.191 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.191 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T10:41:05.191 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.191 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.191 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.191 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2024-04-21T10:41:05.192 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2024-04-21T10:41:05.192 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.192 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.192 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.192 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.192 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.192 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.192 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.193 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T10:41:05.193 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.193 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.193 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.193 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2024-04-21T10:41:05.193 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2024-04-21T10:41:05.193 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.194 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.194 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.194 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.194 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.194 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.194 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.194 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.194 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.195 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.195 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.195 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2024-04-21T10:41:05.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2024-04-21T10:41:05.195 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.195 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.195 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.196 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.196 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.196 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.196 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.196 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.196 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.196 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.196 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.197 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2024-04-21T10:41:05.197 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2024-04-21T10:41:05.197 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.197 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.197 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.197 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T10:41:05.197 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.198 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.198 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.198 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.198 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.198 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.198 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.199 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2024-04-21T10:41:05.199 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2024-04-21T10:41:05.199 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.199 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.199 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.199 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.199 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.199 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.199 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.200 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.200 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.200 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.200 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.200 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:05.200 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:05.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:05.201 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.201 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.201 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2024-04-21T10:41:05.201 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2024-04-21T10:41:05.201 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.201 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.201 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.201 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.201 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.201 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.202 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2024-04-21T10:41:05.202 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2024-04-21T10:41:05.202 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:05.202 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.202 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.202 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2024-04-21T10:41:05.202 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.203 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.203 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.203 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.203 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.203 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.203 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.203 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2024-04-21T10:41:05.203 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2024-04-21T10:41:05.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.204 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.204 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.204 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.204 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.204 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.204 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.204 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.204 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.205 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.205 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.205 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2024-04-21T10:41:05.205 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2024-04-21T10:41:05.205 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.205 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.205 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.205 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.206 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.206 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.206 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.206 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.206 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.206 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.206 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.206 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.207 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.207 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.207 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.207 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.207 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.207 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.207 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.207 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.207 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.208 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.208 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.208 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.208 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.208 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.208 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.208 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.208 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.208 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.209 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.209 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.209 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.209 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.209 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.209 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.209 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.210 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.210 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.210 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.210 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.210 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.210 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.210 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.210 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.211 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.211 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.211 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.211 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.211 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.211 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.211 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.212 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.212 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.212 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.212 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.212 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.212 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.212 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.212 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.213 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.213 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.213 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.213 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.213 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.213 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.213 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.213 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.214 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.214 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.214 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.214 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.214 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.214 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.214 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.214 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.215 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.215 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2024-04-21T10:41:05.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2024-04-21T10:41:05.215 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.215 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.215 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.215 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.215 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.216 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.216 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.216 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.216 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.216 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.216 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.216 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.216 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2024-04-21T10:41:05.216 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2024-04-21T10:41:05.217 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.217 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.217 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.217 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.217 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.217 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.217 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.217 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.217 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.218 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.218 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.218 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2024-04-21T10:41:05.218 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2024-04-21T10:41:05.218 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.218 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.218 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.219 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T10:41:05.219 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.219 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.219 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.219 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.219 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.219 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.219 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.219 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.220 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.220 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.220 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2024-04-21T10:41:05.220 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.220 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.220 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.220 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.221 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.221 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.221 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.221 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.221 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.221 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.221 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.221 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.221 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.222 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.222 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.222 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.222 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.222 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.222 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.222 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.222 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.223 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.223 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.223 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.223 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.223 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.223 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.223 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.223 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.223 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.223 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.224 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.224 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.224 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.224 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.224 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.224 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.224 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.224 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.225 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.225 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.225 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.225 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.225 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.225 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.225 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.225 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.225 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.226 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.226 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.226 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.226 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.226 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.226 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.226 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.226 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.227 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.227 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.227 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.227 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.227 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.227 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.227 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.227 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.228 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.228 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.228 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.228 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.228 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.228 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.228 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.228 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.229 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-04-21T10:41:05.229 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.229 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.229 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.229 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.229 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.229 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.229 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.229 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.230 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.230 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.230 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.230 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.230 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.230 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.230 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.230 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.231 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.231 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.231 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.231 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.231 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.231 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.231 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.231 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.232 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.232 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.232 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.232 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.232 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.232 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.232 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.232 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.232 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.233 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.233 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.233 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.233 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.233 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.233 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.233 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.233 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.233 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.234 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.234 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.234 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.234 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.234 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.234 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.234 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.234 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.235 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.235 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.235 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.235 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.235 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.235 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.235 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.235 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.235 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.236 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.236 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.236 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.236 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.236 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.236 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.236 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.236 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.236 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.237 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.237 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.237 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.237 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.237 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2024-04-21T10:41:05.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2024-04-21T10:41:05.237 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:05.237 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.237 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.238 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2024-04-21T10:41:05.238 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.238 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.238 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.238 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T10:41:05.238 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.238 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.238 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.239 INFO:teuthology.orchestra.run.smithi026.stdout: "periodic_async_work": { 2024-04-21T10:41:05.239 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "periodic_async_work", 2024-04-21T10:41:05.239 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.239 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.239 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.239 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.239 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.239 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.240 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.240 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Periodically check for async work", 2024-04-21T10:41:05.240 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.240 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.240 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.240 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2024-04-21T10:41:05.240 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2024-04-21T10:41:05.240 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:05.241 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.241 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.241 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2024-04-21T10:41:05.241 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.241 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.241 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.241 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T10:41:05.241 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.241 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.242 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.242 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_no_wait": { 2024-04-21T10:41:05.242 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T10:41:05.242 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.242 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.242 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.242 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2024-04-21T10:41:05.243 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.243 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.243 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.243 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T10:41:05.243 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.243 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.243 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.243 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.244 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.244 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.244 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2024-04-21T10:41:05.244 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2024-04-21T10:41:05.244 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2024-04-21T10:41:05.244 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2024-04-21T10:41:05.244 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2024-04-21T10:41:05.245 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2024-04-21T10:41:05.245 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2024-04-21T10:41:05.245 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.245 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.245 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2024-04-21T10:41:05.245 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.245 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.245 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.246 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.246 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.246 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.246 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.246 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2024-04-21T10:41:05.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2024-04-21T10:41:05.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.247 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.247 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.247 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.247 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.247 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.247 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.247 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.247 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.247 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.248 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.248 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2024-04-21T10:41:05.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2024-04-21T10:41:05.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2024-04-21T10:41:05.248 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.248 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.248 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2024-04-21T10:41:05.248 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.249 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.249 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.249 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.249 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.249 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.249 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.249 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2024-04-21T10:41:05.250 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2024-04-21T10:41:05.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.250 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.250 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.250 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.250 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.250 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.250 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.251 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.251 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.251 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.251 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.251 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.251 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.251 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.251 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.252 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.252 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.252 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.252 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2024-04-21T10:41:05.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2024-04-21T10:41:05.252 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.252 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.253 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.253 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.253 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.253 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.253 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.253 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.253 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.253 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.254 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.254 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2024-04-21T10:41:05.254 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2024-04-21T10:41:05.254 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.254 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.254 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.254 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2024-04-21T10:41:05.255 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.255 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.255 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2024-04-21T10:41:05.255 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2024-04-21T10:41:05.255 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2024-04-21T10:41:05.255 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2024-04-21T10:41:05.255 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2024-04-21T10:41:05.255 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2024-04-21T10:41:05.255 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2024-04-21T10:41:05.256 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.256 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.256 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.256 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.256 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.256 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2024-04-21T10:41:05.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2024-04-21T10:41:05.257 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2024-04-21T10:41:05.257 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.257 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2024-04-21T10:41:05.257 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2024-04-21T10:41:05.257 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.257 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.258 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.258 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.258 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.258 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.258 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.258 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2024-04-21T10:41:05.258 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2024-04-21T10:41:05.259 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.259 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.259 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.259 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2024-04-21T10:41:05.259 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.259 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.259 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.259 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.259 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.260 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.260 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.260 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2024-04-21T10:41:05.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2024-04-21T10:41:05.260 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2024-04-21T10:41:05.260 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.260 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.260 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2024-04-21T10:41:05.261 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.261 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.261 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.261 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.261 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.261 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.261 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.262 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2024-04-21T10:41:05.262 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2024-04-21T10:41:05.262 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2024-04-21T10:41:05.262 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2024-04-21T10:41:05.262 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2024-04-21T10:41:05.262 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T10:41:05.262 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2024-04-21T10:41:05.262 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2024-04-21T10:41:05.263 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2024-04-21T10:41:05.263 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2024-04-21T10:41:05.263 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2024-04-21T10:41:05.263 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2024-04-21T10:41:05.263 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2024-04-21T10:41:05.263 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.263 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.263 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.263 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.264 INFO:teuthology.orchestra.run.smithi026.stdout: "services": { 2024-04-21T10:41:05.264 INFO:teuthology.orchestra.run.smithi026.stdout: "dashboard": "https://172.21.15.26:8443/" 2024-04-21T10:41:05.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.264 INFO:teuthology.orchestra.run.smithi026.stdout: "always_on_modules": { 2024-04-21T10:41:05.264 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus": [ 2024-04-21T10:41:05.264 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-04-21T10:41:05.264 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-04-21T10:41:05.264 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-04-21T10:41:05.264 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-04-21T10:41:05.265 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-04-21T10:41:05.265 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-04-21T10:41:05.265 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-04-21T10:41:05.265 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-04-21T10:41:05.265 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-04-21T10:41:05.265 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-04-21T10:41:05.265 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.265 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific": [ 2024-04-21T10:41:05.265 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-04-21T10:41:05.266 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-04-21T10:41:05.266 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-04-21T10:41:05.266 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-04-21T10:41:05.266 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-04-21T10:41:05.266 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-04-21T10:41:05.266 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-04-21T10:41:05.266 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-04-21T10:41:05.266 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-04-21T10:41:05.266 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-04-21T10:41:05.267 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.267 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy": [ 2024-04-21T10:41:05.267 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-04-21T10:41:05.267 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-04-21T10:41:05.267 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-04-21T10:41:05.267 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-04-21T10:41:05.267 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-04-21T10:41:05.267 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-04-21T10:41:05.267 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-04-21T10:41:05.268 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-04-21T10:41:05.268 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-04-21T10:41:05.268 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-04-21T10:41:05.268 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.268 INFO:teuthology.orchestra.run.smithi026.stdout: "reef": [ 2024-04-21T10:41:05.268 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-04-21T10:41:05.268 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-04-21T10:41:05.268 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-04-21T10:41:05.268 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-04-21T10:41:05.268 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-04-21T10:41:05.269 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-04-21T10:41:05.269 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-04-21T10:41:05.269 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-04-21T10:41:05.269 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-04-21T10:41:05.269 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-04-21T10:41:05.269 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-21T10:41:05.269 INFO:teuthology.orchestra.run.smithi026.stdout: "squid": [ 2024-04-21T10:41:05.269 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2024-04-21T10:41:05.269 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2024-04-21T10:41:05.270 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2024-04-21T10:41:05.270 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2024-04-21T10:41:05.270 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2024-04-21T10:41:05.270 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2024-04-21T10:41:05.270 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2024-04-21T10:41:05.270 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2024-04-21T10:41:05.270 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2024-04-21T10:41:05.270 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2024-04-21T10:41:05.270 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-21T10:41:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: "last_failure_osd_epoch": 90, 2024-04-21T10:41:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: "active_clients": [ 2024-04-21T10:41:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2024-04-21T10:41:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-04-21T10:41:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-04-21T10:41:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-04-21T10:41:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 245597547 2024-04-21T10:41:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-21T10:41:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "libcephsqlite", 2024-04-21T10:41:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-04-21T10:41:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.272 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-04-21T10:41:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-04-21T10:41:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 730864061 2024-04-21T10:41:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-21T10:41:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2024-04-21T10:41:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-04-21T10:41:05.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-04-21T10:41:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-04-21T10:41:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 102166939 2024-04-21T10:41:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-21T10:41:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-04-21T10:41:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-04-21T10:41:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-04-21T10:41:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-04-21T10:41:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1317334720 2024-04-21T10:41:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-21T10:41:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.275 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2024-04-21T10:41:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-04-21T10:41:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-04-21T10:41:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-04-21T10:41:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1849640031 2024-04-21T10:41:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-21T10:41:05.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2024-04-21T10:41:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-04-21T10:41:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-21T10:41:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2024-04-21T10:41:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2024-04-21T10:41:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 2175981475 2024-04-21T10:41:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.277 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-21T10:41:05.278 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:05.278 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-21T10:41:05.278 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:41:05.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:03 smithi190 ceph-mon[31423]: pgmap v313: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:41:05.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:03 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1528130518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:41:05.279 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:03 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1161065628' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T10:41:05.280 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:03 smithi169 ceph-mon[31469]: pgmap v313: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:41:05.280 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:03 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1528130518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:41:05.280 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:03 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1161065628' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T10:41:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.944+0000 7f9485523640 1 -- 172.21.15.26:0/1161065628 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f94680a3d20 msgr2=0x7f94680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.944+0000 7f9485523640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f94680a3d20 0x7f94680a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9470005fd0 tx=0x7f947000d5b0 comp rx=0 tx=0).stop 2024-04-21T10:41:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.944+0000 7f9485523640 1 -- 172.21.15.26:0/1161065628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9480101260 msgr2=0x7f9480198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.944+0000 7f9485523640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9480101260 0x7f9480198c70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9474005940 tx=0x7f947402d890 comp rx=0 tx=0).stop 2024-04-21T10:41:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.944+0000 7f9485523640 1 -- 172.21.15.26:0/1161065628 shutdown_connections 2024-04-21T10:41:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.944+0000 7f9485523640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f94680a3d20 0x7f94680a61e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.944+0000 7f9485523640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9480106710 0x7f948019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.944+0000 7f9485523640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9480101260 0x7f9480198c70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:05.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.944+0000 7f9485523640 1 --2- 172.21.15.26:0/1161065628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94801008a0 0x7f9480198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:05.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.944+0000 7f9485523640 1 -- 172.21.15.26:0/1161065628 >> 172.21.15.26:0/1161065628 conn(0x7f9480076130 msgr2=0x7f94800ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:05.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.944+0000 7f9485523640 1 -- 172.21.15.26:0/1161065628 shutdown_connections 2024-04-21T10:41:05.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:02.944+0000 7f9485523640 1 -- 172.21.15.26:0/1161065628 wait complete. 2024-04-21T10:41:05.288 DEBUG:tasks.check_counter:Getting stats from a 2024-04-21T10:41:05.289 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-21T10:41:05.289 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:03 smithi026 ceph-mon[27965]: pgmap v313: 97 pgs: 97 active+clean; 65 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:41:05.289 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:03 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1528130518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:41:05.289 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:03 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1161065628' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T10:41:05.547 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:41:05.669 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:05 smithi026 ceph-mon[27965]: pgmap v314: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-04-21T10:41:05.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:05 smithi169 ceph-mon[31469]: pgmap v314: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-04-21T10:41:05.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:05 smithi190 ceph-mon[31423]: pgmap v314: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-04-21T10:41:07.248 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:41:07.249 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T10:41:07.249 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 212, 2024-04-21T10:41:07.249 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 200, 2024-04-21T10:41:07.249 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 1450416, 2024-04-21T10:41:07.249 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 121952, 2024-04-21T10:41:07.249 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-04-21T10:41:07.250 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-04-21T10:41:07.250 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.041516815, 2024-04-21T10:41:07.250 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.013276221, 2024-04-21T10:41:07.250 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.031287973, 2024-04-21T10:41:07.250 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.004002147, 2024-04-21T10:41:07.250 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:07.251 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 200, 2024-04-21T10:41:07.251 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.010464044, 2024-04-21T10:41:07.251 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000052320 2024-04-21T10:41:07.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:07.251 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:07.251 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:07.252 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:07.252 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:07.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:07.252 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1450416, 2024-04-21T10:41:07.252 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 121952 2024-04-21T10:41:07.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:07.253 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T10:41:07.253 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 2, 2024-04-21T10:41:07.253 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 151, 2024-04-21T10:41:07.253 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 320, 2024-04-21T10:41:07.253 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 182784, 2024-04-21T10:41:07.253 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2024-04-21T10:41:07.253 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2024-04-21T10:41:07.254 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.013034645, 2024-04-21T10:41:07.254 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.008727764, 2024-04-21T10:41:07.254 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.001913205, 2024-04-21T10:41:07.254 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T10:41:07.254 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:07.254 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 148, 2024-04-21T10:41:07.254 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.010861848, 2024-04-21T10:41:07.255 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000073390 2024-04-21T10:41:07.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:07.255 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:07.256 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:07.256 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:07.256 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:07.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:07.256 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T10:41:07.256 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 182784 2024-04-21T10:41:07.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:07.257 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T10:41:07.257 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2024-04-21T10:41:07.257 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2024-04-21T10:41:07.257 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2024-04-21T10:41:07.257 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2024-04-21T10:41:07.257 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-04-21T10:41:07.257 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-04-21T10:41:07.258 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.000459806, 2024-04-21T10:41:07.258 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T10:41:07.258 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000394436, 2024-04-21T10:41:07.258 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T10:41:07.258 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:07.258 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:07.258 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:07.259 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:07.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:07.259 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:07.259 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:07.259 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:07.259 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:07.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:07.259 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T10:41:07.260 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T10:41:07.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:07.260 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-04-21T10:41:07.260 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-04-21T10:41:07.260 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-04-21T10:41:07.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:07.260 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-04-21T10:41:07.261 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-04-21T10:41:07.261 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-04-21T10:41:07.261 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T10:41:07.261 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-04-21T10:41:07.261 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T10:41:07.261 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-04-21T10:41:07.261 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T10:41:07.261 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T10:41:07.262 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T10:41:07.262 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T10:41:07.262 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T10:41:07.262 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-04-21T10:41:07.262 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T10:41:07.262 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T10:41:07.263 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-04-21T10:41:07.263 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-04-21T10:41:07.263 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-04-21T10:41:07.263 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-04-21T10:41:07.263 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T10:41:07.263 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T10:41:07.264 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T10:41:07.264 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T10:41:07.264 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T10:41:07.264 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-04-21T10:41:07.264 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-04-21T10:41:07.264 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-04-21T10:41:07.264 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T10:41:07.265 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T10:41:07.265 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-04-21T10:41:07.265 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-04-21T10:41:07.265 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-04-21T10:41:07.265 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-04-21T10:41:07.265 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T10:41:07.265 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-04-21T10:41:07.265 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T10:41:07.266 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-04-21T10:41:07.266 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 149007, 2024-04-21T10:41:07.266 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 30, 2024-04-21T10:41:07.266 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-04-21T10:41:07.266 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-04-21T10:41:07.266 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-04-21T10:41:07.266 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-04-21T10:41:07.267 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-04-21T10:41:07.267 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-04-21T10:41:07.267 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-04-21T10:41:07.267 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-04-21T10:41:07.267 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-04-21T10:41:07.267 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-04-21T10:41:07.267 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T10:41:07.268 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-04-21T10:41:07.268 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-04-21T10:41:07.268 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-04-21T10:41:07.268 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-04-21T10:41:07.268 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-04-21T10:41:07.268 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-04-21T10:41:07.268 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-04-21T10:41:07.268 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-04-21T10:41:07.269 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-04-21T10:41:07.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:07.269 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T10:41:07.269 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-04-21T10:41:07.269 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-04-21T10:41:07.269 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-04-21T10:41:07.269 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 210, 2024-04-21T10:41:07.270 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1418040, 2024-04-21T10:41:07.270 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:07.270 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 210, 2024-04-21T10:41:07.270 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-04-21T10:41:07.270 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-04-21T10:41:07.270 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 210, 2024-04-21T10:41:07.270 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1418040, 2024-04-21T10:41:07.271 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-04-21T10:41:07.271 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:07.271 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:07.271 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:07.271 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:07.271 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:07.271 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:41:07.550 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:07 smithi026 ceph-mon[27965]: pgmap v315: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-21T10:41:07.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:07 smithi169 ceph-mon[31469]: pgmap v315: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-21T10:41:07.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:07 smithi190 ceph-mon[31423]: pgmap v315: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-21T10:41:07.942 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-21T10:41:07.942 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-21T10:41:07.943 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-21T10:41:07.943 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-21T10:41:07.943 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-21T10:41:07.943 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-21T10:41:07.943 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-21T10:41:07.943 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-21T10:41:07.943 DEBUG:tasks.check_counter:Getting stats from d 2024-04-21T10:41:07.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-21T10:41:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:41:09.633 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:09 smithi026 ceph-mon[27965]: pgmap v316: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T10:41:09.633 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:09 smithi026 ceph-mon[27965]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T10:41:09.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:09 smithi169 ceph-mon[31469]: pgmap v316: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T10:41:09.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:09 smithi169 ceph-mon[31469]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T10:41:09.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:09 smithi190 ceph-mon[31423]: pgmap v316: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-21T10:41:09.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:09 smithi190 ceph-mon[31423]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T10:41:09.863 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:41:09.864 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T10:41:09.864 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2024-04-21T10:41:09.864 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2024-04-21T10:41:09.864 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2024-04-21T10:41:09.864 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2024-04-21T10:41:09.864 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-04-21T10:41:09.864 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-04-21T10:41:09.865 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.001394146, 2024-04-21T10:41:09.865 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T10:41:09.865 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000341569, 2024-04-21T10:41:09.865 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T10:41:09.865 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:09.865 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:09.865 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:09.865 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:09.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:09.866 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:09.866 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:09.866 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:09.866 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:09.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:09.866 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T10:41:09.866 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T10:41:09.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:09.867 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T10:41:09.867 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 213, 2024-04-21T10:41:09.867 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 203, 2024-04-21T10:41:09.867 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 1448736, 2024-04-21T10:41:09.867 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 124704, 2024-04-21T10:41:09.867 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-04-21T10:41:09.867 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-04-21T10:41:09.868 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.042631483, 2024-04-21T10:41:09.868 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.013449059, 2024-04-21T10:41:09.868 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.033370869, 2024-04-21T10:41:09.868 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.004030629, 2024-04-21T10:41:09.868 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:09.868 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 202, 2024-04-21T10:41:09.868 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.009353627, 2024-04-21T10:41:09.868 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000046305 2024-04-21T10:41:09.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:09.869 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:09.869 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:09.869 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:09.869 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:09.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:09.869 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1448736, 2024-04-21T10:41:09.869 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 124704 2024-04-21T10:41:09.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:09.869 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T10:41:09.870 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 6, 2024-04-21T10:41:09.870 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 158, 2024-04-21T10:41:09.870 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 3360, 2024-04-21T10:41:09.870 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 184112, 2024-04-21T10:41:09.870 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2024-04-21T10:41:09.870 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2024-04-21T10:41:09.870 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.014462267, 2024-04-21T10:41:09.870 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.009632463, 2024-04-21T10:41:09.871 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.002464728, 2024-04-21T10:41:09.871 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T10:41:09.871 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:09.871 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 156, 2024-04-21T10:41:09.871 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.014989854, 2024-04-21T10:41:09.871 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000096088 2024-04-21T10:41:09.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:09.871 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:09.871 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:09.872 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:09.872 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:09.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:09.872 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-21T10:41:09.872 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 184112 2024-04-21T10:41:09.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:09.872 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-04-21T10:41:09.872 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-04-21T10:41:09.873 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-04-21T10:41:09.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:09.873 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-04-21T10:41:09.873 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-04-21T10:41:09.873 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-04-21T10:41:09.873 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T10:41:09.873 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-04-21T10:41:09.873 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T10:41:09.873 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-04-21T10:41:09.874 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T10:41:09.874 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T10:41:09.874 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T10:41:09.874 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T10:41:09.874 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T10:41:09.874 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-04-21T10:41:09.874 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T10:41:09.874 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T10:41:09.875 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-04-21T10:41:09.875 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-04-21T10:41:09.875 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-04-21T10:41:09.875 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-04-21T10:41:09.875 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T10:41:09.875 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T10:41:09.875 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T10:41:09.875 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T10:41:09.876 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T10:41:09.876 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-04-21T10:41:09.876 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-04-21T10:41:09.876 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-04-21T10:41:09.876 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T10:41:09.876 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T10:41:09.876 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-04-21T10:41:09.876 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-04-21T10:41:09.876 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-04-21T10:41:09.877 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-04-21T10:41:09.877 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T10:41:09.877 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-04-21T10:41:09.877 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T10:41:09.877 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-04-21T10:41:09.877 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 149017, 2024-04-21T10:41:09.877 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 31, 2024-04-21T10:41:09.878 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-04-21T10:41:09.878 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-04-21T10:41:09.878 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-04-21T10:41:09.878 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-04-21T10:41:09.878 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-04-21T10:41:09.878 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-04-21T10:41:09.878 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-04-21T10:41:09.879 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-04-21T10:41:09.879 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-04-21T10:41:09.879 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-04-21T10:41:09.879 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T10:41:09.879 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-04-21T10:41:09.879 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-04-21T10:41:09.879 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-04-21T10:41:09.879 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-04-21T10:41:09.879 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-04-21T10:41:09.880 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-04-21T10:41:09.880 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-04-21T10:41:09.880 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-04-21T10:41:09.880 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-04-21T10:41:09.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:09.880 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T10:41:09.880 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-04-21T10:41:09.880 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-04-21T10:41:09.880 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-04-21T10:41:09.881 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 215, 2024-04-21T10:41:09.881 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1418690, 2024-04-21T10:41:09.881 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:09.881 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 215, 2024-04-21T10:41:09.881 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-04-21T10:41:09.881 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-04-21T10:41:09.881 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 215, 2024-04-21T10:41:09.882 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1418690, 2024-04-21T10:41:09.882 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-04-21T10:41:09.882 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:09.882 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:09.882 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:09.882 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:09.882 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:09.882 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:41:10.543 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-21T10:41:10.543 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-21T10:41:10.543 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-21T10:41:10.543 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-21T10:41:10.543 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-21T10:41:10.543 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-21T10:41:10.544 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-21T10:41:10.544 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-21T10:41:10.544 DEBUG:tasks.check_counter:Getting stats from g 2024-04-21T10:41:10.544 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-21T10:41:10.795 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:41:11.618 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:11 smithi026 ceph-mon[27965]: pgmap v318: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-21T10:41:11.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:11 smithi169 ceph-mon[31469]: pgmap v318: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-21T10:41:11.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:11 smithi190 ceph-mon[31423]: pgmap v318: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-21T10:41:12.526 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:12 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:41:12.602 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:12 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:41:12.603 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:41:12.603 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T10:41:12.603 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2024-04-21T10:41:12.603 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2024-04-21T10:41:12.604 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2024-04-21T10:41:12.604 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2024-04-21T10:41:12.604 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-04-21T10:41:12.604 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-04-21T10:41:12.604 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.001468580, 2024-04-21T10:41:12.604 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T10:41:12.604 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000395068, 2024-04-21T10:41:12.604 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T10:41:12.605 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:12.605 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:12.605 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:12.605 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:12.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:12.605 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:12.606 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:12.606 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:12.606 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:12.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:12.606 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T10:41:12.606 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T10:41:12.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:12.606 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T10:41:12.607 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 6, 2024-04-21T10:41:12.607 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 154, 2024-04-21T10:41:12.607 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 3360, 2024-04-21T10:41:12.607 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 183408, 2024-04-21T10:41:12.607 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2024-04-21T10:41:12.607 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2024-04-21T10:41:12.607 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.013653787, 2024-04-21T10:41:12.607 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.008848322, 2024-04-21T10:41:12.608 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.002461613, 2024-04-21T10:41:12.608 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T10:41:12.608 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:12.608 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 152, 2024-04-21T10:41:12.608 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.015056730, 2024-04-21T10:41:12.608 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000099057 2024-04-21T10:41:12.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:12.608 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:12.608 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:12.609 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:12.609 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:12.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:12.609 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-21T10:41:12.609 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 183408 2024-04-21T10:41:12.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:12.609 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T10:41:12.609 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 210, 2024-04-21T10:41:12.610 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 200, 2024-04-21T10:41:12.610 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 1447920, 2024-04-21T10:41:12.610 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 122784, 2024-04-21T10:41:12.610 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-04-21T10:41:12.610 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-04-21T10:41:12.610 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.041730394, 2024-04-21T10:41:12.610 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.012887884, 2024-04-21T10:41:12.611 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.033444640, 2024-04-21T10:41:12.611 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.004236645, 2024-04-21T10:41:12.611 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:12.611 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 199, 2024-04-21T10:41:12.611 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.010055379, 2024-04-21T10:41:12.611 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000050529 2024-04-21T10:41:12.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:12.612 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:12.612 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:12.612 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:12.612 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:12.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:12.613 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1447920, 2024-04-21T10:41:12.613 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 122784 2024-04-21T10:41:12.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:12.613 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-04-21T10:41:12.613 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-04-21T10:41:12.614 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-04-21T10:41:12.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:12.614 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-04-21T10:41:12.614 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-04-21T10:41:12.615 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-04-21T10:41:12.615 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T10:41:12.615 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-04-21T10:41:12.615 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T10:41:12.615 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-04-21T10:41:12.616 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T10:41:12.616 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T10:41:12.616 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T10:41:12.616 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T10:41:12.616 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T10:41:12.617 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-04-21T10:41:12.617 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T10:41:12.617 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T10:41:12.617 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-04-21T10:41:12.617 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-04-21T10:41:12.618 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-04-21T10:41:12.618 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-04-21T10:41:12.618 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T10:41:12.618 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T10:41:12.618 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T10:41:12.619 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T10:41:12.619 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T10:41:12.619 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-04-21T10:41:12.619 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-04-21T10:41:12.619 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-04-21T10:41:12.620 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T10:41:12.620 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T10:41:12.620 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-04-21T10:41:12.620 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-04-21T10:41:12.620 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-04-21T10:41:12.621 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-04-21T10:41:12.621 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T10:41:12.621 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-04-21T10:41:12.621 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T10:41:12.621 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-04-21T10:41:12.622 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148977, 2024-04-21T10:41:12.622 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 30, 2024-04-21T10:41:12.622 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-04-21T10:41:12.622 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-04-21T10:41:12.622 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-04-21T10:41:12.623 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-04-21T10:41:12.623 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-04-21T10:41:12.623 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-04-21T10:41:12.623 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-04-21T10:41:12.624 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-04-21T10:41:12.624 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-04-21T10:41:12.624 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-04-21T10:41:12.624 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T10:41:12.624 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-04-21T10:41:12.624 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-04-21T10:41:12.624 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-04-21T10:41:12.624 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-04-21T10:41:12.624 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-04-21T10:41:12.625 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-04-21T10:41:12.625 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-04-21T10:41:12.625 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-04-21T10:41:12.625 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-04-21T10:41:12.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:12.625 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T10:41:12.625 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-04-21T10:41:12.626 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-04-21T10:41:12.626 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-04-21T10:41:12.626 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 212, 2024-04-21T10:41:12.626 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1418300, 2024-04-21T10:41:12.626 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:12.626 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 212, 2024-04-21T10:41:12.626 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-04-21T10:41:12.626 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-04-21T10:41:12.627 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 212, 2024-04-21T10:41:12.627 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1418300, 2024-04-21T10:41:12.627 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-04-21T10:41:12.627 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:12.627 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:12.627 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:12.627 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:12.627 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:12.627 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:41:12.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:12 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T10:41:13.263 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-21T10:41:13.263 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-21T10:41:13.263 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-21T10:41:13.264 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-21T10:41:13.264 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-21T10:41:13.264 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-21T10:41:13.264 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-21T10:41:13.264 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-21T10:41:13.265 DEBUG:tasks.check_counter:Getting stats from j 2024-04-21T10:41:13.265 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-21T10:41:13.513 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:13 smithi026 ceph-mon[27965]: pgmap v319: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-21T10:41:13.513 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:13 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:13.513 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:13 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:13.513 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:13 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:13.514 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:13 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:13.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:13 smithi169 ceph-mon[31469]: pgmap v319: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-21T10:41:13.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:13 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:13.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:13 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:13.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:13 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:13.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:13 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:13.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:13 smithi190 ceph-mon[31423]: pgmap v319: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-21T10:41:13.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:13 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:13.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:13 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:13.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:13 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:13.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:13 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:13.982 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T10:41:14.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:14 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:14 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:14 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:14 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:14 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-21T10:41:14.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:14 smithi026 ceph-mon[27965]: Unable to set osd_memory_target on smithi169 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-21T10:41:14.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:14 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:14 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:14 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:14 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.686 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:14 smithi026 ceph-mon[27965]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T10:41:14.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:14 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:14 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:14 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:14 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:14 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-21T10:41:14.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:14 smithi169 ceph-mon[31469]: Unable to set osd_memory_target on smithi169 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-21T10:41:14.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:14 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:14 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:14 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:14 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:14 smithi169 ceph-mon[31469]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T10:41:14.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:14 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:14 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:14 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:14 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:14 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-21T10:41:14.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:14 smithi190 ceph-mon[31423]: Unable to set osd_memory_target on smithi169 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-21T10:41:14.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:14 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:14 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:14 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:14 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:14.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:14 smithi190 ceph-mon[31423]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T10:41:15.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:15 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi190 to 176.2M 2024-04-21T10:41:15.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:15 smithi026 ceph-mon[27965]: Unable to set osd_memory_target on smithi190 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-21T10:41:15.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:15 smithi026 ceph-mon[27965]: pgmap v321: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-21T10:41:15.678 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:15 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi190 to 176.2M 2024-04-21T10:41:15.678 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:15 smithi190 ceph-mon[31423]: Unable to set osd_memory_target on smithi190 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-21T10:41:15.678 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:15 smithi190 ceph-mon[31423]: pgmap v321: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-21T10:41:15.689 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-21T10:41:15.689 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T10:41:15.689 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 216, 2024-04-21T10:41:15.689 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 206, 2024-04-21T10:41:15.689 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 1449552, 2024-04-21T10:41:15.689 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 126624, 2024-04-21T10:41:15.689 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-04-21T10:41:15.690 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2024-04-21T10:41:15.690 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.048221181, 2024-04-21T10:41:15.690 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.014647289, 2024-04-21T10:41:15.690 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.036638528, 2024-04-21T10:41:15.690 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.004347535, 2024-04-21T10:41:15.690 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:15.690 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 205, 2024-04-21T10:41:15.690 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.011293734, 2024-04-21T10:41:15.691 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000055091 2024-04-21T10:41:15.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:15.691 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:15.691 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:15.691 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:15.691 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:15.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:15.691 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1449552, 2024-04-21T10:41:15.691 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 126624 2024-04-21T10:41:15.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:15.692 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T10:41:15.692 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 2, 2024-04-21T10:41:15.692 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 157, 2024-04-21T10:41:15.693 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 320, 2024-04-21T10:41:15.693 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 183840, 2024-04-21T10:41:15.693 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2024-04-21T10:41:15.693 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2024-04-21T10:41:15.693 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.014399441, 2024-04-21T10:41:15.693 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.009837794, 2024-04-21T10:41:15.693 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.002054676, 2024-04-21T10:41:15.693 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T10:41:15.693 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:15.694 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 154, 2024-04-21T10:41:15.694 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.010497422, 2024-04-21T10:41:15.694 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000068165 2024-04-21T10:41:15.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:15.694 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:15.694 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:15.694 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:15.694 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:15.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:15.695 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T10:41:15.695 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 183840 2024-04-21T10:41:15.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:15.695 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T10:41:15.695 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 4, 2024-04-21T10:41:15.695 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 4, 2024-04-21T10:41:15.695 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 3040, 2024-04-21T10:41:15.695 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 448, 2024-04-21T10:41:15.696 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2024-04-21T10:41:15.696 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2024-04-21T10:41:15.696 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.000653041, 2024-04-21T10:41:15.696 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000040286, 2024-04-21T10:41:15.696 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000602197, 2024-04-21T10:41:15.696 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T10:41:15.696 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:15.696 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3, 2024-04-21T10:41:15.697 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000722824, 2024-04-21T10:41:15.697 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000240941 2024-04-21T10:41:15.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:15.697 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:15.697 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:15.697 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:15.697 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:15.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:15.697 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-21T10:41:15.698 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T10:41:15.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:15.698 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-04-21T10:41:15.698 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2024-04-21T10:41:15.698 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-04-21T10:41:15.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:15.698 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-04-21T10:41:15.698 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-04-21T10:41:15.699 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-04-21T10:41:15.699 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T10:41:15.699 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-04-21T10:41:15.699 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T10:41:15.699 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-04-21T10:41:15.699 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T10:41:15.699 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T10:41:15.699 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T10:41:15.699 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T10:41:15.700 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T10:41:15.700 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-04-21T10:41:15.700 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T10:41:15.700 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T10:41:15.700 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-04-21T10:41:15.700 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-04-21T10:41:15.700 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-04-21T10:41:15.700 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-04-21T10:41:15.700 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T10:41:15.701 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T10:41:15.701 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T10:41:15.701 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T10:41:15.701 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T10:41:15.701 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-04-21T10:41:15.701 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-04-21T10:41:15.701 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-04-21T10:41:15.701 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T10:41:15.702 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T10:41:15.702 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-04-21T10:41:15.702 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-04-21T10:41:15.702 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-04-21T10:41:15.702 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-04-21T10:41:15.702 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T10:41:15.702 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-04-21T10:41:15.702 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T10:41:15.702 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-04-21T10:41:15.703 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148977, 2024-04-21T10:41:15.703 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 30, 2024-04-21T10:41:15.703 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-04-21T10:41:15.703 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-04-21T10:41:15.703 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-04-21T10:41:15.703 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-04-21T10:41:15.703 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-04-21T10:41:15.703 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-04-21T10:41:15.704 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-04-21T10:41:15.704 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-04-21T10:41:15.704 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2024-04-21T10:41:15.704 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2024-04-21T10:41:15.704 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T10:41:15.704 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-04-21T10:41:15.704 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-04-21T10:41:15.704 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-04-21T10:41:15.705 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2024-04-21T10:41:15.705 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2024-04-21T10:41:15.705 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-04-21T10:41:15.705 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-04-21T10:41:15.705 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-04-21T10:41:15.705 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-04-21T10:41:15.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-21T10:41:15.705 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T10:41:15.705 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-04-21T10:41:15.706 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-04-21T10:41:15.706 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-04-21T10:41:15.706 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 218, 2024-04-21T10:41:15.706 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1419080, 2024-04-21T10:41:15.706 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:15.706 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 218, 2024-04-21T10:41:15.706 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-04-21T10:41:15.706 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-04-21T10:41:15.707 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 218, 2024-04-21T10:41:15.707 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1419080, 2024-04-21T10:41:15.707 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-04-21T10:41:15.707 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-04-21T10:41:15.707 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-04-21T10:41:15.707 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-04-21T10:41:15.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:15.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-21T10:41:15.707 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-21T10:41:15.708 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:15 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi190 to 176.2M 2024-04-21T10:41:15.708 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:15 smithi169 ceph-mon[31469]: Unable to set osd_memory_target on smithi190 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-21T10:41:15.708 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:15 smithi169 ceph-mon[31469]: pgmap v321: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 5 op/s 2024-04-21T10:41:16.297 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-21T10:41:16.297 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-21T10:41:16.298 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-21T10:41:16.298 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-21T10:41:16.298 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-21T10:41:16.298 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-21T10:41:16.298 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-21T10:41:16.298 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-21T10:41:16.298 DEBUG:tasks.check_counter:Getting stats from b 2024-04-21T10:41:16.299 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-21T10:41:16.551 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T10:41:17.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:17 smithi190 ceph-mon[31423]: pgmap v322: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:41:17.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:17 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:17.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:17 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:17.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:17 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:17.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:17 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:17.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:17 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:17.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:17 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:17.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:17 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:41:17.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:17 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:41:17.778 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:17 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:17.816 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:17 smithi169 ceph-mon[31469]: pgmap v322: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:41:17.816 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:17 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:17.816 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:17 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:17.816 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:17 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:17.816 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:17 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:17.817 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:17 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:17.817 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:17 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:17.817 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:17 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:41:17.817 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:17 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:41:17.817 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:17 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:17.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:17 smithi026 ceph-mon[27965]: pgmap v322: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-21T10:41:17.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:17 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:17.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:17 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:17.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:17 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:17.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:17 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:17.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:17 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:17.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:17 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T10:41:17.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:17 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T10:41:17.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:17 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T10:41:17.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:17 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' 2024-04-21T10:41:18.555 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-21T10:41:18.555 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T10:41:18.555 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 916, 2024-04-21T10:41:18.555 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 843, 2024-04-21T10:41:18.555 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 307579, 2024-04-21T10:41:18.555 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 473061, 2024-04-21T10:41:18.555 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 10, 2024-04-21T10:41:18.556 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 5, 2024-04-21T10:41:18.556 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.081461187, 2024-04-21T10:41:18.556 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.032208153, 2024-04-21T10:41:18.556 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.044689054, 2024-04-21T10:41:18.556 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.006191984, 2024-04-21T10:41:18.556 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:18.556 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 842, 2024-04-21T10:41:18.556 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.046462083, 2024-04-21T10:41:18.557 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000055180 2024-04-21T10:41:18.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.557 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:18.557 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1608, 2024-04-21T10:41:18.557 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000380201, 2024-04-21T10:41:18.557 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000236 2024-04-21T10:41:18.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.557 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 4768, 2024-04-21T10:41:18.558 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 6320 2024-04-21T10:41:18.558 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.558 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T10:41:18.558 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 193, 2024-04-21T10:41:18.558 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 266, 2024-04-21T10:41:18.558 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 49758, 2024-04-21T10:41:18.558 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 250637, 2024-04-21T10:41:18.558 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 17, 2024-04-21T10:41:18.559 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 5, 2024-04-21T10:41:18.559 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.029626862, 2024-04-21T10:41:18.559 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.009906736, 2024-04-21T10:41:18.559 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.021255302, 2024-04-21T10:41:18.559 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000437427, 2024-04-21T10:41:18.559 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:18.559 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 263, 2024-04-21T10:41:18.559 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.021695043, 2024-04-21T10:41:18.560 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000082490 2024-04-21T10:41:18.560 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.560 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:18.560 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 374, 2024-04-21T10:41:18.560 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000059895, 2024-04-21T10:41:18.560 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000160 2024-04-21T10:41:18.560 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.560 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-04-21T10:41:18.561 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 128240 2024-04-21T10:41:18.561 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.561 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T10:41:18.561 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1142, 2024-04-21T10:41:18.561 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 910, 2024-04-21T10:41:18.561 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1843447, 2024-04-21T10:41:18.561 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 552203, 2024-04-21T10:41:18.561 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 7, 2024-04-21T10:41:18.562 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 6, 2024-04-21T10:41:18.562 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.105216626, 2024-04-21T10:41:18.562 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.039065827, 2024-04-21T10:41:18.562 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.079308535, 2024-04-21T10:41:18.562 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.006534360, 2024-04-21T10:41:18.562 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:18.562 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 909, 2024-04-21T10:41:18.562 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.049803271, 2024-04-21T10:41:18.563 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000054789 2024-04-21T10:41:18.563 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.563 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:18.563 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1371, 2024-04-21T10:41:18.563 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000217556, 2024-04-21T10:41:18.563 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000158 2024-04-21T10:41:18.563 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.563 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1513968, 2024-04-21T10:41:18.563 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 440736 2024-04-21T10:41:18.563 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.564 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-21T10:41:18.564 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-21T10:41:18.564 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-21T10:41:18.564 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.564 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-21T10:41:18.564 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-21T10:41:18.564 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-21T10:41:18.564 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 94, 2024-04-21T10:41:18.565 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.009206342, 2024-04-21T10:41:18.565 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000097939 2024-04-21T10:41:18.565 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:18.565 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.565 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-21T10:41:18.565 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-21T10:41:18.565 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-21T10:41:18.565 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 24, 2024-04-21T10:41:18.565 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000734416, 2024-04-21T10:41:18.566 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000030600 2024-04-21T10:41:18.566 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:18.566 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.566 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-21T10:41:18.566 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 130, 2024-04-21T10:41:18.566 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 13, 2024-04-21T10:41:18.566 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-21T10:41:18.566 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 13, 2024-04-21T10:41:18.567 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.014656569, 2024-04-21T10:41:18.567 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001127428 2024-04-21T10:41:18.567 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.567 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-21T10:41:18.567 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 117, 2024-04-21T10:41:18.567 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-21T10:41:18.567 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-21T10:41:18.567 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 11, 2024-04-21T10:41:18.568 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-21T10:41:18.568 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-21T10:41:18.568 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 56, 2024-04-21T10:41:18.568 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 9, 2024-04-21T10:41:18.568 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 47, 2024-04-21T10:41:18.568 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-21T10:41:18.568 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 39, 2024-04-21T10:41:18.568 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 10, 2024-04-21T10:41:18.569 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 1, 2024-04-21T10:41:18.569 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 1, 2024-04-21T10:41:18.569 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 6, 2024-04-21T10:41:18.569 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 402, 2024-04-21T10:41:18.569 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 244, 2024-04-21T10:41:18.569 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 117, 2024-04-21T10:41:18.569 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 29, 2024-04-21T10:41:18.569 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-21T10:41:18.570 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-21T10:41:18.570 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-21T10:41:18.570 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-21T10:41:18.570 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-21T10:41:18.570 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-21T10:41:18.570 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-21T10:41:18.570 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 2, 2024-04-21T10:41:18.571 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 16, 2024-04-21T10:41:18.571 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-21T10:41:18.571 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-21T10:41:18.571 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-21T10:41:18.571 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 4, 2024-04-21T10:41:18.571 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 234, 2024-04-21T10:41:18.571 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-21T10:41:18.571 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T10:41:18.572 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-21T10:41:18.572 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T10:41:18.572 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-21T10:41:18.572 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-21T10:41:18.572 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T10:41:18.572 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T10:41:18.572 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-21T10:41:18.572 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-21T10:41:18.573 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 2, 2024-04-21T10:41:18.573 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 10, 2024-04-21T10:41:18.573 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T10:41:18.573 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T10:41:18.573 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T10:41:18.573 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 17, 2024-04-21T10:41:18.573 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 12, 2024-04-21T10:41:18.573 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 7, 2024-04-21T10:41:18.573 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 8 2024-04-21T10:41:18.573 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.574 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-21T10:41:18.574 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 1, 2024-04-21T10:41:18.574 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 469, 2024-04-21T10:41:18.574 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 247, 2024-04-21T10:41:18.574 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 446, 2024-04-21T10:41:18.574 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 1, 2024-04-21T10:41:18.574 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-21T10:41:18.574 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-21T10:41:18.575 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-21T10:41:18.575 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 4, 2024-04-21T10:41:18.575 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 4, 2024-04-21T10:41:18.575 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-21T10:41:18.575 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-21T10:41:18.575 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-21T10:41:18.575 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-21T10:41:18.575 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-21T10:41:18.575 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-21T10:41:18.576 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-21T10:41:18.576 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 2, 2024-04-21T10:41:18.576 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 11, 2024-04-21T10:41:18.576 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T10:41:18.576 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-21T10:41:18.576 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-21T10:41:18.576 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-21T10:41:18.576 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-21T10:41:18.576 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-21T10:41:18.577 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.577 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-21T10:41:18.577 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-21T10:41:18.577 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 44, 2024-04-21T10:41:18.577 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-21T10:41:18.577 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-21T10:41:18.577 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 44, 2024-04-21T10:41:18.577 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-21T10:41:18.577 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-21T10:41:18.578 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 2, 2024-04-21T10:41:18.578 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-21T10:41:18.578 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-21T10:41:18.578 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-04-21T10:41:18.578 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-04-21T10:41:18.578 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-21T10:41:18.578 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-21T10:41:18.578 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-04-21T10:41:18.579 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4268296, 2024-04-21T10:41:18.579 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-04-21T10:41:18.579 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-21T10:41:18.579 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 20, 2024-04-21T10:41:18.579 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.115641931, 2024-04-21T10:41:18.579 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.005782096 2024-04-21T10:41:18.579 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.579 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-04-21T10:41:18.579 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.580 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-21T10:41:18.580 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 51, 2024-04-21T10:41:18.580 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 61, 2024-04-21T10:41:18.580 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 10, 2024-04-21T10:41:18.580 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 47, 2024-04-21T10:41:18.580 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 248, 2024-04-21T10:41:18.580 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 201, 2024-04-21T10:41:18.580 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 56, 2024-04-21T10:41:18.581 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 70, 2024-04-21T10:41:18.581 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 14, 2024-04-21T10:41:18.581 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 1, 2024-04-21T10:41:18.581 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 13, 2024-04-21T10:41:18.581 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 12, 2024-04-21T10:41:18.581 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 54092, 2024-04-21T10:41:18.581 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 223608 2024-04-21T10:41:18.581 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.581 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-21T10:41:18.582 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 141, 2024-04-21T10:41:18.582 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 1, 2024-04-21T10:41:18.582 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 130, 2024-04-21T10:41:18.582 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 26, 2024-04-21T10:41:18.582 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 18, 2024-04-21T10:41:18.582 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-21T10:41:18.582 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-04-21T10:41:18.582 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.003555979, 2024-04-21T10:41:18.582 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.003555979 2024-04-21T10:41:18.583 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.583 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-21T10:41:18.583 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.583 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.583 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.583 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.583 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-21T10:41:18.583 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.584 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.584 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.584 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.584 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-21T10:41:18.584 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.584 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.584 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.584 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.585 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-21T10:41:18.585 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 7, 2024-04-21T10:41:18.585 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.002351307, 2024-04-21T10:41:18.585 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000335901 2024-04-21T10:41:18.585 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.585 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-21T10:41:18.585 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.585 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.586 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.586 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.586 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-21T10:41:18.586 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.586 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.586 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.586 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.586 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-21T10:41:18.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.587 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.587 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.587 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-21T10:41:18.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.587 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.588 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.588 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-21T10:41:18.588 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.588 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.588 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.588 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.588 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-21T10:41:18.588 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.588 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.589 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.589 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.589 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-21T10:41:18.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.590 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.590 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.590 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-21T10:41:18.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.590 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.591 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.591 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-21T10:41:18.591 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.591 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.591 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.591 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.591 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-21T10:41:18.591 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-04-21T10:41:18.592 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000330145, 2024-04-21T10:41:18.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000330145 2024-04-21T10:41:18.592 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.592 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-21T10:41:18.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.592 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.592 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.593 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-21T10:41:18.593 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.593 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.593 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.593 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.593 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-21T10:41:18.593 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.593 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.593 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.594 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.594 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-21T10:41:18.594 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.594 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.594 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.594 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.594 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-21T10:41:18.595 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.595 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.595 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.595 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.595 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-21T10:41:18.595 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.595 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.595 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.595 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.596 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-21T10:41:18.596 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.596 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.596 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.596 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.596 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-21T10:41:18.596 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.596 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.597 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.597 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.597 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-21T10:41:18.597 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.597 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.597 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.597 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.597 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-21T10:41:18.598 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.598 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.598 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.598 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.598 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-21T10:41:18.598 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.598 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.598 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.598 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.599 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-21T10:41:18.599 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.599 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.599 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.599 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.599 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-21T10:41:18.599 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.599 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.600 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.600 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.600 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-21T10:41:18.600 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4, 2024-04-21T10:41:18.600 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.008419138, 2024-04-21T10:41:18.600 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002104784 2024-04-21T10:41:18.600 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.600 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-21T10:41:18.601 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-21T10:41:18.601 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-21T10:41:18.601 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.601 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.601 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.601 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:18.601 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.601 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-21T10:41:18.602 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 2, 2024-04-21T10:41:18.602 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 4, 2024-04-21T10:41:18.602 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 2, 2024-04-21T10:41:18.602 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 2, 2024-04-21T10:41:18.602 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-21T10:41:18.602 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 12, 2024-04-21T10:41:18.602 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 3, 2024-04-21T10:41:18.602 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 584, 2024-04-21T10:41:18.602 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-21T10:41:18.603 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.603 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-21T10:41:18.603 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-21T10:41:18.603 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-21T10:41:18.603 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T10:41:18.603 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-21T10:41:18.603 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T10:41:18.603 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-21T10:41:18.604 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T10:41:18.604 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T10:41:18.604 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T10:41:18.604 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T10:41:18.604 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T10:41:18.604 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-21T10:41:18.604 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T10:41:18.604 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T10:41:18.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-21T10:41:18.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-21T10:41:18.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-21T10:41:18.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-21T10:41:18.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T10:41:18.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T10:41:18.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T10:41:18.605 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T10:41:18.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T10:41:18.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-21T10:41:18.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-21T10:41:18.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-21T10:41:18.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T10:41:18.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T10:41:18.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-21T10:41:18.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-21T10:41:18.606 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-21T10:41:18.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-21T10:41:18.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T10:41:18.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-21T10:41:18.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T10:41:18.607 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-21T10:41:18.607 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 286339, 2024-04-21T10:41:18.607 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 136, 2024-04-21T10:41:18.607 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 88, 2024-04-21T10:41:18.608 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 1, 2024-04-21T10:41:18.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-21T10:41:18.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-21T10:41:18.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-21T10:41:18.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-21T10:41:18.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-21T10:41:18.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-21T10:41:18.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 8136, 2024-04-21T10:41:18.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 170, 2024-04-21T10:41:18.609 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T10:41:18.609 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-21T10:41:18.609 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-21T10:41:18.609 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-21T10:41:18.609 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 227291, 2024-04-21T10:41:18.609 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 1496, 2024-04-21T10:41:18.609 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-21T10:41:18.609 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-21T10:41:18.610 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-21T10:41:18.610 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-21T10:41:18.610 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.610 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-21T10:41:18.610 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-21T10:41:18.610 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-21T10:41:18.610 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 87, 2024-04-21T10:41:18.610 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 87924, 2024-04-21T10:41:18.611 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 33, 2024-04-21T10:41:18.611 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 87, 2024-04-21T10:41:18.611 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-21T10:41:18.611 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 87, 2024-04-21T10:41:18.611 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.596986862, 2024-04-21T10:41:18.611 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.006861917 2024-04-21T10:41:18.611 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.611 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-21T10:41:18.611 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-21T10:41:18.612 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 87, 2024-04-21T10:41:18.612 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 120 2024-04-21T10:41:18.612 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.612 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 87, 2024-04-21T10:41:18.612 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 2, 2024-04-21T10:41:18.612 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 85, 2024-04-21T10:41:18.612 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-21T10:41:18.612 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-21T10:41:18.613 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 0, 2024-04-21T10:41:18.613 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 10, 2024-04-21T10:41:18.613 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 2, 2024-04-21T10:41:18.613 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 22, 2024-04-21T10:41:18.613 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 13, 2024-04-21T10:41:18.613 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-21T10:41:18.613 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-21T10:41:18.613 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-21T10:41:18.614 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-21T10:41:18.614 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 16, 2024-04-21T10:41:18.614 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-21T10:41:18.614 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-21T10:41:18.614 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-21T10:41:18.614 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 0, 2024-04-21T10:41:18.614 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 20, 2024-04-21T10:41:18.614 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-21T10:41:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-21T10:41:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-21T10:41:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-21T10:41:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-21T10:41:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-21T10:41:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T10:41:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-21T10:41:18.615 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-21T10:41:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-21T10:41:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-21T10:41:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-21T10:41:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-21T10:41:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-21T10:41:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-21T10:41:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-21T10:41:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-21T10:41:18.616 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-21T10:41:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-21T10:41:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-21T10:41:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-21T10:41:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-21T10:41:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-21T10:41:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-21T10:41:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-21T10:41:18.617 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-21T10:41:18.618 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 100, 2024-04-21T10:41:18.618 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-21T10:41:18.618 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 13, 2024-04-21T10:41:18.618 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 10, 2024-04-21T10:41:18.618 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 10, 2024-04-21T10:41:18.618 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-21T10:41:18.618 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-21T10:41:18.618 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 31, 2024-04-21T10:41:18.619 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-04-21T10:41:18.619 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 6 2024-04-21T10:41:18.619 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.619 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-21T10:41:18.619 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-04-21T10:41:18.619 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-21T10:41:18.619 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-21T10:41:18.619 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-21T10:41:18.620 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.620 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-21T10:41:18.620 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 12 2024-04-21T10:41:18.620 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.620 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-21T10:41:18.620 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-21T10:41:18.620 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 3, 2024-04-21T10:41:18.620 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-21T10:41:18.620 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 3, 2024-04-21T10:41:18.621 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 4, 2024-04-21T10:41:18.621 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 4, 2024-04-21T10:41:18.621 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-21T10:41:18.621 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.621 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T10:41:18.621 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:18.621 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-21T10:41:18.621 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:18.622 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 2247, 2024-04-21T10:41:18.622 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1981181, 2024-04-21T10:41:18.622 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:18.622 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 2247, 2024-04-21T10:41:18.622 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-21T10:41:18.622 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-21T10:41:18.622 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2247, 2024-04-21T10:41:18.623 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1981181, 2024-04-21T10:41:18.623 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:18.623 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.623 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.623 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.623 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:18.623 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.623 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-21T10:41:18.623 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:18.624 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-21T10:41:18.624 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:18.624 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-21T10:41:18.624 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-21T10:41:18.624 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:18.624 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:18.625 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 87, 2024-04-21T10:41:18.625 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 88328, 2024-04-21T10:41:18.625 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 87, 2024-04-21T10:41:18.625 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 88328, 2024-04-21T10:41:18.625 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:18.625 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.625 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.625 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.626 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:18.626 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.626 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-21T10:41:18.626 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:18.626 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-21T10:41:18.626 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:18.626 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-21T10:41:18.626 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-21T10:41:18.627 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:18.627 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:18.627 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 87, 2024-04-21T10:41:18.627 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 87, 2024-04-21T10:41:18.627 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 87, 2024-04-21T10:41:18.627 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 87, 2024-04-21T10:41:18.627 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:18.627 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.627 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.628 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.628 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:18.628 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.628 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-21T10:41:18.628 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:18.628 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-21T10:41:18.628 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:18.628 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 4, 2024-04-21T10:41:18.628 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 404, 2024-04-21T10:41:18.629 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:18.629 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 4, 2024-04-21T10:41:18.629 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-21T10:41:18.629 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-21T10:41:18.629 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2, 2024-04-21T10:41:18.629 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 404, 2024-04-21T10:41:18.629 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:18.629 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.630 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.630 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.630 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:18.630 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:18.630 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x561a759ec500": { 2024-04-21T10:41:18.630 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:18.630 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-21T10:41:18.630 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:18.631 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 44, 2024-04-21T10:41:18.631 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 73992, 2024-04-21T10:41:18.631 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:18.631 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 44, 2024-04-21T10:41:18.631 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-21T10:41:18.631 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-21T10:41:18.631 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 20, 2024-04-21T10:41:18.631 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 73992, 2024-04-21T10:41:18.631 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:18.632 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:18.632 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:18.632 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:18.632 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:18.632 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:18.632 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-21T10:41:18.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:18 smithi190 ceph-mon[31423]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-04-21T10:41:18.777 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:18 smithi190 ceph-mon[31423]: Unable to set osd_memory_target on smithi026 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-21T10:41:18.922 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:18 smithi169 ceph-mon[31469]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-04-21T10:41:18.922 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:18 smithi169 ceph-mon[31469]: Unable to set osd_memory_target on smithi026 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-21T10:41:18.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:18 smithi026 ceph-mon[27965]: Adjusting osd_memory_target on smithi026 to 177.2M 2024-04-21T10:41:18.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:18 smithi026 ceph-mon[27965]: Unable to set osd_memory_target on smithi026 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-21T10:41:19.341 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-21T10:41:19.341 INFO:tasks.check_counter:Daemon mds.b mds.imported=2 2024-04-21T10:41:19.341 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1 2024-04-21T10:41:19.341 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=469 2024-04-21T10:41:19.342 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=0 2024-04-21T10:41:19.342 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-04-21T10:41:19.342 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T10:41:19.342 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-04-21T10:41:19.342 DEBUG:tasks.check_counter:Getting stats from e 2024-04-21T10:41:19.342 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-21T10:41:19.590 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T10:41:19.620 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:19 smithi169 ceph-mon[31469]: pgmap v323: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 213 B/s wr, 6 op/s 2024-04-21T10:41:19.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:19 smithi190 ceph-mon[31423]: pgmap v323: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 213 B/s wr, 6 op/s 2024-04-21T10:41:19.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:19 smithi026 ceph-mon[27965]: pgmap v323: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 213 B/s wr, 6 op/s 2024-04-21T10:41:21.121 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-21T10:41:21.121 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T10:41:21.121 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 238, 2024-04-21T10:41:21.121 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 240, 2024-04-21T10:41:21.122 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 12649329, 2024-04-21T10:41:21.122 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 75748, 2024-04-21T10:41:21.122 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 7, 2024-04-21T10:41:21.122 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-04-21T10:41:21.122 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.028468312, 2024-04-21T10:41:21.122 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.007745548, 2024-04-21T10:41:21.122 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.012341841, 2024-04-21T10:41:21.122 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.005608032, 2024-04-21T10:41:21.123 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:21.123 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 238, 2024-04-21T10:41:21.123 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.009756234, 2024-04-21T10:41:21.123 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000040992 2024-04-21T10:41:21.123 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.123 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:21.123 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.123 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.123 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.124 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.124 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T10:41:21.124 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T10:41:21.124 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.124 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T10:41:21.124 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 423, 2024-04-21T10:41:21.124 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 406, 2024-04-21T10:41:21.124 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 18324475, 2024-04-21T10:41:21.125 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 187625, 2024-04-21T10:41:21.125 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-04-21T10:41:21.125 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-04-21T10:41:21.125 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.075450053, 2024-04-21T10:41:21.125 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.018572656, 2024-04-21T10:41:21.125 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.053497036, 2024-04-21T10:41:21.125 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.011309386, 2024-04-21T10:41:21.125 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:21.126 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 405, 2024-04-21T10:41:21.126 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.019365824, 2024-04-21T10:41:21.126 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000047816 2024-04-21T10:41:21.126 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.126 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:21.126 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.126 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.126 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.127 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.127 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1501472, 2024-04-21T10:41:21.127 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 129744 2024-04-21T10:41:21.127 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.127 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T10:41:21.127 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1807, 2024-04-21T10:41:21.127 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1960, 2024-04-21T10:41:21.127 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 9047917, 2024-04-21T10:41:21.127 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 1011502, 2024-04-21T10:41:21.128 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 8, 2024-04-21T10:41:21.128 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 6, 2024-04-21T10:41:21.128 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.190830605, 2024-04-21T10:41:21.128 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.055863959, 2024-04-21T10:41:21.128 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.053753590, 2024-04-21T10:41:21.128 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.064586562, 2024-04-21T10:41:21.128 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:21.128 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1958, 2024-04-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.085536892, 2024-04-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000043685 2024-04-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 438784 2024-04-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3717, 2024-04-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.117288698, 2024-04-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000031554 2024-04-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 0, 2024-04-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 0, 2024-04-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-21T10:41:21.133 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.133 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.133 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.133 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.133 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-21T10:41:21.133 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-04-21T10:41:21.133 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-21T10:41:21.133 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-21T10:41:21.133 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 0, 2024-04-21T10:41:21.134 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-21T10:41:21.134 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-21T10:41:21.134 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 5714, 2024-04-21T10:41:21.134 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 189, 2024-04-21T10:41:21.134 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 5525, 2024-04-21T10:41:21.134 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-21T10:41:21.135 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 5445, 2024-04-21T10:41:21.135 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-04-21T10:41:21.135 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 0, 2024-04-21T10:41:21.135 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 0, 2024-04-21T10:41:21.135 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 71, 2024-04-21T10:41:21.135 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 0, 2024-04-21T10:41:21.135 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 0, 2024-04-21T10:41:21.135 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-04-21T10:41:21.136 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-04-21T10:41:21.136 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-21T10:41:21.136 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-21T10:41:21.136 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-21T10:41:21.136 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-21T10:41:21.136 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-21T10:41:21.136 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-21T10:41:21.136 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-21T10:41:21.137 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-04-21T10:41:21.137 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-04-21T10:41:21.137 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-21T10:41:21.137 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-21T10:41:21.137 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-21T10:41:21.137 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 0, 2024-04-21T10:41:21.138 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 0, 2024-04-21T10:41:21.138 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-21T10:41:21.138 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T10:41:21.138 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-21T10:41:21.138 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T10:41:21.138 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-21T10:41:21.138 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-21T10:41:21.138 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T10:41:21.139 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T10:41:21.139 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-21T10:41:21.139 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-21T10:41:21.139 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T10:41:21.139 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-04-21T10:41:21.139 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T10:41:21.139 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T10:41:21.139 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T10:41:21.140 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-04-21T10:41:21.140 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-04-21T10:41:21.140 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-04-21T10:41:21.140 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-04-21T10:41:21.140 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.140 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-21T10:41:21.140 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-04-21T10:41:21.140 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-04-21T10:41:21.141 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-04-21T10:41:21.141 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-04-21T10:41:21.141 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-04-21T10:41:21.141 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-21T10:41:21.141 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-21T10:41:21.141 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-21T10:41:21.141 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-04-21T10:41:21.141 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-04-21T10:41:21.142 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-21T10:41:21.142 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-21T10:41:21.142 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-21T10:41:21.142 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-21T10:41:21.142 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-21T10:41:21.142 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-21T10:41:21.142 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-21T10:41:21.142 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-21T10:41:21.142 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-21T10:41:21.143 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T10:41:21.143 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-21T10:41:21.143 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-21T10:41:21.143 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-21T10:41:21.143 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-21T10:41:21.143 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-21T10:41:21.143 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.143 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-21T10:41:21.144 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-21T10:41:21.144 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 0, 2024-04-21T10:41:21.144 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-21T10:41:21.144 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-21T10:41:21.144 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 17567, 2024-04-21T10:41:21.144 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-21T10:41:21.144 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-21T10:41:21.144 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 0, 2024-04-21T10:41:21.144 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-21T10:41:21.145 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-21T10:41:21.145 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 24, 2024-04-21T10:41:21.145 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 3, 2024-04-21T10:41:21.145 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-21T10:41:21.145 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-21T10:41:21.145 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-04-21T10:41:21.145 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 42080911, 2024-04-21T10:41:21.145 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 42080337, 2024-04-21T10:41:21.145 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-21T10:41:21.145 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.146 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.146 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.146 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.146 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 17567 2024-04-21T10:41:21.146 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.146 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-21T10:41:21.146 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 2326, 2024-04-21T10:41:21.147 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 6131, 2024-04-21T10:41:21.147 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 3805, 2024-04-21T10:41:21.147 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 245, 2024-04-21T10:41:21.147 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 278, 2024-04-21T10:41:21.147 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 33, 2024-04-21T10:41:21.147 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 5714, 2024-04-21T10:41:21.147 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 7282, 2024-04-21T10:41:21.147 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 1568, 2024-04-21T10:41:21.147 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 0, 2024-04-21T10:41:21.148 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 0, 2024-04-21T10:41:21.148 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 0, 2024-04-21T10:41:21.148 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 66104, 2024-04-21T10:41:21.148 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 215416 2024-04-21T10:41:21.148 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.148 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-21T10:41:21.148 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 0, 2024-04-21T10:41:21.148 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-04-21T10:41:21.148 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 0, 2024-04-21T10:41:21.149 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 0, 2024-04-21T10:41:21.149 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-04-21T10:41:21.149 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-21T10:41:21.149 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.149 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.149 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.149 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.149 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-21T10:41:21.150 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.150 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.150 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.150 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.150 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-21T10:41:21.150 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.150 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.150 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.150 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.151 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-21T10:41:21.151 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.151 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.151 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.151 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.153 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-21T10:41:21.153 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.153 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.153 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.153 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.153 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-21T10:41:21.153 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.153 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.154 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.154 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.154 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-21T10:41:21.154 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.154 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.154 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.155 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.155 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-21T10:41:21.155 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.155 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.155 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.155 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.155 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-21T10:41:21.155 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.155 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.156 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.156 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.156 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-21T10:41:21.156 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.156 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.156 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.156 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.156 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-21T10:41:21.156 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.157 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.157 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.157 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.157 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-21T10:41:21.158 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.158 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.158 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.158 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.158 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-21T10:41:21.158 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.158 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.158 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.159 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.159 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-21T10:41:21.159 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.159 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.159 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.159 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.159 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-21T10:41:21.159 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.159 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.160 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.160 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.160 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-21T10:41:21.160 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.160 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.160 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.160 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.160 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-21T10:41:21.160 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.161 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.161 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.161 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.161 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-21T10:41:21.161 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.161 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.161 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.161 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.161 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-21T10:41:21.161 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.162 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.162 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.162 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.162 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-21T10:41:21.162 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.162 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.162 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.162 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.163 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-21T10:41:21.163 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.163 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.163 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.163 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.163 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-21T10:41:21.163 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.163 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.163 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.164 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.164 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-21T10:41:21.164 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.164 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.164 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.164 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.164 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-21T10:41:21.164 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.164 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.164 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.165 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.165 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-21T10:41:21.165 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.165 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.165 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.165 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.165 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-21T10:41:21.165 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.165 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.166 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.166 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.166 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-21T10:41:21.166 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.166 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.166 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.166 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.166 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-21T10:41:21.167 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.167 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.167 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.167 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.167 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-21T10:41:21.167 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.167 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.167 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.168 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.168 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-21T10:41:21.168 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-21T10:41:21.168 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-21T10:41:21.168 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.168 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.168 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.168 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:21.168 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.168 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-21T10:41:21.169 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-04-21T10:41:21.169 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 5, 2024-04-21T10:41:21.169 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 4, 2024-04-21T10:41:21.169 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 4, 2024-04-21T10:41:21.169 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-21T10:41:21.169 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 0, 2024-04-21T10:41:21.169 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 0, 2024-04-21T10:41:21.169 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 5043, 2024-04-21T10:41:21.169 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-21T10:41:21.170 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.170 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-21T10:41:21.170 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-21T10:41:21.170 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-21T10:41:21.170 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T10:41:21.170 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-21T10:41:21.170 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T10:41:21.170 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-21T10:41:21.171 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T10:41:21.171 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T10:41:21.171 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T10:41:21.171 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T10:41:21.171 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T10:41:21.171 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-21T10:41:21.171 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T10:41:21.171 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T10:41:21.171 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-21T10:41:21.172 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-21T10:41:21.172 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-21T10:41:21.172 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-21T10:41:21.172 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T10:41:21.172 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T10:41:21.172 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T10:41:21.172 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T10:41:21.172 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T10:41:21.172 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-21T10:41:21.173 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-21T10:41:21.173 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-21T10:41:21.173 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T10:41:21.173 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T10:41:21.173 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-21T10:41:21.173 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-21T10:41:21.173 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-21T10:41:21.173 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-21T10:41:21.173 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T10:41:21.174 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-21T10:41:21.174 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T10:41:21.174 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-21T10:41:21.174 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 309667, 2024-04-21T10:41:21.174 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 2398, 2024-04-21T10:41:21.174 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-21T10:41:21.174 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-21T10:41:21.174 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-21T10:41:21.174 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-21T10:41:21.175 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-21T10:41:21.175 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-21T10:41:21.175 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-21T10:41:21.175 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-21T10:41:21.175 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 8136, 2024-04-21T10:41:21.175 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 170, 2024-04-21T10:41:21.175 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T10:41:21.175 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-21T10:41:21.175 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-21T10:41:21.176 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-21T10:41:21.176 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 9698537, 2024-04-21T10:41:21.176 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 81071, 2024-04-21T10:41:21.176 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-21T10:41:21.176 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-21T10:41:21.176 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-21T10:41:21.176 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-21T10:41:21.176 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.176 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-21T10:41:21.177 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-21T10:41:21.177 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-21T10:41:21.177 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 2221, 2024-04-21T10:41:21.177 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 21, 2024-04-21T10:41:21.177 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 3, 2024-04-21T10:41:21.177 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 2221, 2024-04-21T10:41:21.177 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-21T10:41:21.177 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2221, 2024-04-21T10:41:21.177 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1.259972009, 2024-04-21T10:41:21.178 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000567299 2024-04-21T10:41:21.178 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.178 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-21T10:41:21.178 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-21T10:41:21.178 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2221, 2024-04-21T10:41:21.178 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 2222 2024-04-21T10:41:21.178 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.178 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 2221, 2024-04-21T10:41:21.179 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 2221, 2024-04-21T10:41:21.179 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 0, 2024-04-21T10:41:21.179 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-21T10:41:21.179 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-21T10:41:21.179 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 1416, 2024-04-21T10:41:21.179 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 0, 2024-04-21T10:41:21.179 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 803, 2024-04-21T10:41:21.180 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 0, 2024-04-21T10:41:21.180 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 0, 2024-04-21T10:41:21.180 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-21T10:41:21.180 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-21T10:41:21.180 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-21T10:41:21.180 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-21T10:41:21.180 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 0, 2024-04-21T10:41:21.181 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-21T10:41:21.181 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-21T10:41:21.181 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-21T10:41:21.181 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 1, 2024-04-21T10:41:21.181 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 0, 2024-04-21T10:41:21.181 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-21T10:41:21.181 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-21T10:41:21.181 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-21T10:41:21.181 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-21T10:41:21.182 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-21T10:41:21.182 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-21T10:41:21.182 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T10:41:21.182 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-21T10:41:21.182 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-21T10:41:21.182 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-21T10:41:21.182 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-21T10:41:21.182 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-21T10:41:21.183 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-21T10:41:21.183 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-21T10:41:21.183 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-21T10:41:21.183 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-21T10:41:21.183 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-21T10:41:21.183 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-21T10:41:21.183 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-21T10:41:21.183 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-21T10:41:21.183 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-21T10:41:21.184 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-21T10:41:21.184 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-21T10:41:21.184 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-21T10:41:21.184 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-21T10:41:21.184 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-21T10:41:21.184 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 100, 2024-04-21T10:41:21.184 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-21T10:41:21.184 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 13, 2024-04-21T10:41:21.185 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 9, 2024-04-21T10:41:21.185 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 9, 2024-04-21T10:41:21.185 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-21T10:41:21.185 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-21T10:41:21.185 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 0, 2024-04-21T10:41:21.185 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 2, 2024-04-21T10:41:21.185 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 0 2024-04-21T10:41:21.185 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.185 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-21T10:41:21.186 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 0, 2024-04-21T10:41:21.186 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-21T10:41:21.186 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-21T10:41:21.186 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-21T10:41:21.186 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.186 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-21T10:41:21.186 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-21T10:41:21.186 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.186 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-21T10:41:21.187 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-21T10:41:21.187 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T10:41:21.187 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-21T10:41:21.187 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-04-21T10:41:21.187 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-04-21T10:41:21.187 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-04-21T10:41:21.187 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-21T10:41:21.188 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.188 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T10:41:21.188 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:21.188 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-21T10:41:21.188 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:21.188 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 2464, 2024-04-21T10:41:21.188 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 39784568, 2024-04-21T10:41:21.188 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:21.188 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 2464, 2024-04-21T10:41:21.189 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-21T10:41:21.189 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-21T10:41:21.189 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2464, 2024-04-21T10:41:21.189 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 39784568, 2024-04-21T10:41:21.189 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:21.189 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.189 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.189 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.189 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:21.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.190 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-21T10:41:21.190 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:21.190 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-21T10:41:21.190 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:21.190 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-21T10:41:21.190 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-21T10:41:21.190 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:21.190 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:21.191 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 2221, 2024-04-21T10:41:21.191 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 37890921, 2024-04-21T10:41:21.191 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2221, 2024-04-21T10:41:21.191 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 37890921, 2024-04-21T10:41:21.191 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:21.191 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.191 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.191 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.191 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:21.192 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.192 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-21T10:41:21.192 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:21.192 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-21T10:41:21.192 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:21.192 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-21T10:41:21.192 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-21T10:41:21.192 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:21.193 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:21.193 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 2221, 2024-04-21T10:41:21.193 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 2221, 2024-04-21T10:41:21.193 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2221, 2024-04-21T10:41:21.193 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 2221, 2024-04-21T10:41:21.193 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:21.193 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.193 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.194 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.194 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:21.194 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.194 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-21T10:41:21.194 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:21.194 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-21T10:41:21.194 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:21.194 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-21T10:41:21.194 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-21T10:41:21.195 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:21.195 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:21.195 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-21T10:41:21.195 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-21T10:41:21.195 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-21T10:41:21.195 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-21T10:41:21.195 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:21.195 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.195 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.196 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.196 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:21.196 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:21.196 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x55708762c6e0": { 2024-04-21T10:41:21.196 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:21.196 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-21T10:41:21.196 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:21.197 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-21T10:41:21.197 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-21T10:41:21.197 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:21.197 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:21.197 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-21T10:41:21.197 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-21T10:41:21.197 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-21T10:41:21.197 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-21T10:41:21.198 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:21.198 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:21.198 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:21.198 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:21.198 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:21.198 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:21.198 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-21T10:41:21.697 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:21 smithi169 ceph-mon[31469]: pgmap v324: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:41:21.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:21 smithi190 ceph-mon[31423]: pgmap v324: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:41:21.902 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-21T10:41:21.903 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-21T10:41:21.903 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-21T10:41:21.903 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-21T10:41:21.903 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-21T10:41:21.903 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-21T10:41:21.903 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T10:41:21.903 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-21T10:41:21.903 DEBUG:tasks.check_counter:Getting stats from h 2024-04-21T10:41:21.904 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-21T10:41:21.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:21 smithi026 ceph-mon[27965]: pgmap v324: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:41:22.144 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T10:41:23.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:23 smithi190 ceph-mon[31423]: pgmap v325: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:41:23.783 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:23 smithi169 ceph-mon[31469]: pgmap v325: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:41:23.909 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-21T10:41:23.909 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T10:41:23.909 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 980, 2024-04-21T10:41:23.909 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 961, 2024-04-21T10:41:23.910 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1751600, 2024-04-21T10:41:23.910 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 363278, 2024-04-21T10:41:23.910 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-04-21T10:41:23.910 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 1, 2024-04-21T10:41:23.910 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.126573068, 2024-04-21T10:41:23.910 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.047918925, 2024-04-21T10:41:23.910 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.051468745, 2024-04-21T10:41:23.911 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.026512532, 2024-04-21T10:41:23.911 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:23.911 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 961, 2024-04-21T10:41:23.911 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.051778251, 2024-04-21T10:41:23.911 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000053879 2024-04-21T10:41:23.911 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.911 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:23.911 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.911 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.912 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.912 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.912 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1503968, 2024-04-21T10:41:23.912 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 128912 2024-04-21T10:41:23.912 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.912 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T10:41:23.912 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 4, 2024-04-21T10:41:23.912 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 158, 2024-04-21T10:41:23.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 876, 2024-04-21T10:41:23.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 438858, 2024-04-21T10:41:23.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 6, 2024-04-21T10:41:23.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-04-21T10:41:23.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.010388705, 2024-04-21T10:41:23.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.006350557, 2024-04-21T10:41:23.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.002150851, 2024-04-21T10:41:23.913 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000067237, 2024-04-21T10:41:23.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:23.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 155, 2024-04-21T10:41:23.914 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.013207392, 2024-04-21T10:41:23.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000085208 2024-04-21T10:41:23.914 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.914 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:23.914 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.914 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.915 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T10:41:23.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 438160 2024-04-21T10:41:23.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.915 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T10:41:23.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1381, 2024-04-21T10:41:23.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1383, 2024-04-21T10:41:23.915 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 426155, 2024-04-21T10:41:23.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 439165, 2024-04-21T10:41:23.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-04-21T10:41:23.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-04-21T10:41:23.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.087164863, 2024-04-21T10:41:23.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.020706435, 2024-04-21T10:41:23.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.024740600, 2024-04-21T10:41:23.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.036114173, 2024-04-21T10:41:23.916 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:23.917 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1381, 2024-04-21T10:41:23.917 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.038173668, 2024-04-21T10:41:23.917 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000027642 2024-04-21T10:41:23.917 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.917 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:23.917 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.917 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.917 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.918 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T10:41:23.918 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T10:41:23.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.918 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-21T10:41:23.918 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-21T10:41:23.918 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-21T10:41:23.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.919 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-21T10:41:23.919 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-21T10:41:23.919 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-21T10:41:23.919 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3588, 2024-04-21T10:41:23.919 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.085761994, 2024-04-21T10:41:23.919 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000023902 2024-04-21T10:41:23.919 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:23.919 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.919 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-21T10:41:23.920 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-21T10:41:23.920 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-21T10:41:23.920 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.920 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.920 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.920 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:23.920 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.920 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-21T10:41:23.921 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 0, 2024-04-21T10:41:23.921 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 0, 2024-04-21T10:41:23.921 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-21T10:41:23.921 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.921 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.921 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.921 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.921 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-21T10:41:23.921 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-04-21T10:41:23.922 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-21T10:41:23.922 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-21T10:41:23.922 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 0, 2024-04-21T10:41:23.922 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-21T10:41:23.922 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-21T10:41:23.922 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 30, 2024-04-21T10:41:23.922 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 1, 2024-04-21T10:41:23.922 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 29, 2024-04-21T10:41:23.923 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-21T10:41:23.923 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 29, 2024-04-21T10:41:23.923 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-04-21T10:41:23.923 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 0, 2024-04-21T10:41:23.923 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 0, 2024-04-21T10:41:23.923 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 4, 2024-04-21T10:41:23.923 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 0, 2024-04-21T10:41:23.923 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 0, 2024-04-21T10:41:23.923 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-04-21T10:41:23.924 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-04-21T10:41:23.924 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-21T10:41:23.924 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-21T10:41:23.924 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-21T10:41:23.924 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-21T10:41:23.924 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-21T10:41:23.924 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-21T10:41:23.924 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-21T10:41:23.925 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-04-21T10:41:23.925 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-04-21T10:41:23.925 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-21T10:41:23.925 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-21T10:41:23.925 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-21T10:41:23.925 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 0, 2024-04-21T10:41:23.925 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 0, 2024-04-21T10:41:23.925 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-21T10:41:23.925 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T10:41:23.926 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-21T10:41:23.926 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T10:41:23.926 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-21T10:41:23.926 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-21T10:41:23.926 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T10:41:23.926 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T10:41:23.926 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-21T10:41:23.926 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-21T10:41:23.926 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T10:41:23.927 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-04-21T10:41:23.927 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T10:41:23.927 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T10:41:23.927 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T10:41:23.927 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-04-21T10:41:23.927 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-04-21T10:41:23.927 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-04-21T10:41:23.927 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-04-21T10:41:23.927 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.928 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-21T10:41:23.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-04-21T10:41:23.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-04-21T10:41:23.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-04-21T10:41:23.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-04-21T10:41:23.928 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-04-21T10:41:23.928 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-21T10:41:23.928 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-21T10:41:23.929 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-21T10:41:23.929 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-04-21T10:41:23.929 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-04-21T10:41:23.929 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-21T10:41:23.929 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-21T10:41:23.929 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-21T10:41:23.929 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-21T10:41:23.929 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-21T10:41:23.929 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-21T10:41:23.930 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-21T10:41:23.930 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-21T10:41:23.930 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-21T10:41:23.930 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T10:41:23.930 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-21T10:41:23.930 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-21T10:41:23.930 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-21T10:41:23.930 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-21T10:41:23.931 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-21T10:41:23.931 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.931 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-21T10:41:23.931 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-21T10:41:23.931 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 0, 2024-04-21T10:41:23.931 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-21T10:41:23.931 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-21T10:41:23.931 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 44, 2024-04-21T10:41:23.931 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-21T10:41:23.931 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-21T10:41:23.932 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 0, 2024-04-21T10:41:23.932 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-21T10:41:23.932 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-21T10:41:23.932 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-04-21T10:41:23.932 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-04-21T10:41:23.932 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-21T10:41:23.932 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-21T10:41:23.932 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-04-21T10:41:23.933 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4268296, 2024-04-21T10:41:23.933 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 4267722, 2024-04-21T10:41:23.933 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-21T10:41:23.933 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.933 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.933 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.933 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.933 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 44 2024-04-21T10:41:23.933 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.934 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-21T10:41:23.934 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 21, 2024-04-21T10:41:23.934 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 25, 2024-04-21T10:41:23.934 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 4, 2024-04-21T10:41:23.934 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 12, 2024-04-21T10:41:23.934 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 12, 2024-04-21T10:41:23.934 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 0, 2024-04-21T10:41:23.935 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 30, 2024-04-21T10:41:23.935 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 30, 2024-04-21T10:41:23.935 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 0, 2024-04-21T10:41:23.935 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 0, 2024-04-21T10:41:23.935 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 0, 2024-04-21T10:41:23.935 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 0, 2024-04-21T10:41:23.935 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 50336, 2024-04-21T10:41:23.935 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 215416 2024-04-21T10:41:23.935 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.936 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-21T10:41:23.936 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 0, 2024-04-21T10:41:23.936 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-04-21T10:41:23.936 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 0, 2024-04-21T10:41:23.936 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 0, 2024-04-21T10:41:23.936 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-04-21T10:41:23.936 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-21T10:41:23.936 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.936 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.937 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.937 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-21T10:41:23.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.937 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.937 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.937 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-21T10:41:23.937 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.937 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.938 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.938 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-21T10:41:23.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.938 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.938 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.938 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.938 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-21T10:41:23.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.939 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.939 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-21T10:41:23.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.939 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.939 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.940 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-21T10:41:23.940 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.940 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.940 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.940 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.940 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-21T10:41:23.940 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.940 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.940 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.941 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.941 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-21T10:41:23.941 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.941 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.941 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.941 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.941 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-21T10:41:23.941 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.942 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.942 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.942 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.942 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-21T10:41:23.942 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.942 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.942 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.942 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.942 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-21T10:41:23.943 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:23 smithi026 ceph-mon[27965]: pgmap v325: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:41:23.944 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.944 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.944 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-21T10:41:23.944 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.944 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.944 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.945 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-21T10:41:23.945 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.945 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.945 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.945 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.945 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-21T10:41:23.945 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.945 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.945 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.946 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.946 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-21T10:41:23.946 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.946 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.946 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.946 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.946 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-21T10:41:23.946 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.947 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.947 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.947 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.947 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-21T10:41:23.947 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.947 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.947 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.947 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.948 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-21T10:41:23.948 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.948 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.948 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.948 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.948 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-21T10:41:23.948 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.948 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.948 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.949 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.949 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-21T10:41:23.949 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.949 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.949 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.949 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.949 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-21T10:41:23.949 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.949 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.950 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.950 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.950 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-21T10:41:23.950 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.950 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.950 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.950 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.950 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-21T10:41:23.951 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.951 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.951 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.951 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.951 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-21T10:41:23.951 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.951 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.951 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.951 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.952 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-21T10:41:23.952 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.952 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.952 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.952 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.952 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-21T10:41:23.952 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.952 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.952 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.953 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.953 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-21T10:41:23.953 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.953 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.953 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.953 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.953 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-21T10:41:23.953 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.954 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.954 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.954 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.954 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-21T10:41:23.954 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-21T10:41:23.954 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-21T10:41:23.954 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.954 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.955 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.955 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:23.955 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.955 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-21T10:41:23.955 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 2, 2024-04-21T10:41:23.955 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 4, 2024-04-21T10:41:23.955 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 2, 2024-04-21T10:41:23.955 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 4, 2024-04-21T10:41:23.956 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-21T10:41:23.956 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 0, 2024-04-21T10:41:23.956 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 0, 2024-04-21T10:41:23.956 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 2833, 2024-04-21T10:41:23.956 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-21T10:41:23.956 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.956 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-21T10:41:23.956 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-21T10:41:23.957 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-21T10:41:23.957 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T10:41:23.957 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-21T10:41:23.957 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T10:41:23.957 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-21T10:41:23.957 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T10:41:23.957 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T10:41:23.957 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T10:41:23.958 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T10:41:23.958 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T10:41:23.958 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-21T10:41:23.958 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T10:41:23.958 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T10:41:23.958 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-21T10:41:23.958 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-21T10:41:23.958 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-21T10:41:23.959 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-21T10:41:23.959 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T10:41:23.959 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T10:41:23.959 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T10:41:23.959 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T10:41:23.959 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T10:41:23.959 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-21T10:41:23.959 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-21T10:41:23.960 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-21T10:41:23.960 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T10:41:23.960 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T10:41:23.960 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-21T10:41:23.960 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-21T10:41:23.960 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-21T10:41:23.960 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-21T10:41:23.960 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T10:41:23.961 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-21T10:41:23.961 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T10:41:23.961 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-21T10:41:23.961 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 181861, 2024-04-21T10:41:23.961 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 61, 2024-04-21T10:41:23.961 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-21T10:41:23.961 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-21T10:41:23.961 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-21T10:41:23.962 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-21T10:41:23.962 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-21T10:41:23.962 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-21T10:41:23.962 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-21T10:41:23.962 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-21T10:41:23.962 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 8136, 2024-04-21T10:41:23.962 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 170, 2024-04-21T10:41:23.962 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T10:41:23.963 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-21T10:41:23.963 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-21T10:41:23.963 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-21T10:41:23.963 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 88611, 2024-04-21T10:41:23.963 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 597, 2024-04-21T10:41:23.963 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-21T10:41:23.963 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-21T10:41:23.963 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-21T10:41:23.963 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-21T10:41:23.964 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.964 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-21T10:41:23.964 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-21T10:41:23.964 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-21T10:41:23.964 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 2120, 2024-04-21T10:41:23.964 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 21, 2024-04-21T10:41:23.964 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 3, 2024-04-21T10:41:23.964 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 2120, 2024-04-21T10:41:23.964 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-21T10:41:23.965 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2120, 2024-04-21T10:41:23.965 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1.262971990, 2024-04-21T10:41:23.965 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000595741 2024-04-21T10:41:23.965 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.965 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-21T10:41:23.965 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-21T10:41:23.965 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2120, 2024-04-21T10:41:23.965 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 2121 2024-04-21T10:41:23.965 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.965 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 2120, 2024-04-21T10:41:23.966 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 2120, 2024-04-21T10:41:23.966 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 0, 2024-04-21T10:41:23.966 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-21T10:41:23.966 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-21T10:41:23.966 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 1361, 2024-04-21T10:41:23.966 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 0, 2024-04-21T10:41:23.966 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 757, 2024-04-21T10:41:23.966 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 0, 2024-04-21T10:41:23.967 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 0, 2024-04-21T10:41:23.967 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-21T10:41:23.967 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-21T10:41:23.967 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-21T10:41:23.967 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-21T10:41:23.967 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 0, 2024-04-21T10:41:23.967 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-21T10:41:23.967 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-21T10:41:23.967 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-21T10:41:23.968 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 1, 2024-04-21T10:41:23.968 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 0, 2024-04-21T10:41:23.968 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-21T10:41:23.968 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-21T10:41:23.968 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-21T10:41:23.968 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-21T10:41:23.968 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-21T10:41:23.968 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-21T10:41:23.969 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T10:41:23.969 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-21T10:41:23.969 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-21T10:41:23.969 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-21T10:41:23.969 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-21T10:41:23.969 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-21T10:41:23.969 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-21T10:41:23.969 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-21T10:41:23.969 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-21T10:41:23.969 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-21T10:41:23.970 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-21T10:41:23.970 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-21T10:41:23.970 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-21T10:41:23.970 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-21T10:41:23.970 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-21T10:41:23.970 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-21T10:41:23.970 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-21T10:41:23.970 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-21T10:41:23.971 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-21T10:41:23.971 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-21T10:41:23.971 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 100, 2024-04-21T10:41:23.971 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-21T10:41:23.971 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 13, 2024-04-21T10:41:23.971 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 4, 2024-04-21T10:41:23.971 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 4, 2024-04-21T10:41:23.971 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-21T10:41:23.971 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-21T10:41:23.972 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 0, 2024-04-21T10:41:23.972 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 2, 2024-04-21T10:41:23.972 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 0 2024-04-21T10:41:23.972 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.972 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-21T10:41:23.972 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 0, 2024-04-21T10:41:23.972 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-21T10:41:23.972 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-21T10:41:23.973 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-21T10:41:23.973 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.973 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-21T10:41:23.973 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-21T10:41:23.973 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.973 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-21T10:41:23.973 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-21T10:41:23.973 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T10:41:23.973 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-21T10:41:23.974 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-04-21T10:41:23.974 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-04-21T10:41:23.974 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-04-21T10:41:23.974 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-21T10:41:23.974 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.974 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T10:41:23.974 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:23.974 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-21T10:41:23.975 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:23.975 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 2361, 2024-04-21T10:41:23.975 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1950852, 2024-04-21T10:41:23.975 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:23.975 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 2361, 2024-04-21T10:41:23.975 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-21T10:41:23.975 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-21T10:41:23.975 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2361, 2024-04-21T10:41:23.975 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1950852, 2024-04-21T10:41:23.976 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:23.976 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.976 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.976 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.976 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:23.976 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.976 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-21T10:41:23.976 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:23.976 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-21T10:41:23.977 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:23.977 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-21T10:41:23.977 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-21T10:41:23.977 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:23.977 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:23.977 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 2120, 2024-04-21T10:41:23.977 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 73997, 2024-04-21T10:41:23.977 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2120, 2024-04-21T10:41:23.977 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 73997, 2024-04-21T10:41:23.978 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:23.978 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.978 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.978 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.978 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:23.978 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.978 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-21T10:41:23.978 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:23.979 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-21T10:41:23.979 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:23.979 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-21T10:41:23.979 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-21T10:41:23.979 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:23.979 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:23.979 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 2120, 2024-04-21T10:41:23.979 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 2120, 2024-04-21T10:41:23.979 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2120, 2024-04-21T10:41:23.980 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 2120, 2024-04-21T10:41:23.980 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:23.980 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.980 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.980 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.980 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:23.980 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.980 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-21T10:41:23.981 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:23.981 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-21T10:41:23.981 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:23.981 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-21T10:41:23.981 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-21T10:41:23.981 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:23.981 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:23.981 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-21T10:41:23.981 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-21T10:41:23.981 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-21T10:41:23.982 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-21T10:41:23.982 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:23.982 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.982 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.982 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.982 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:23.982 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:23.983 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x561fea9966e0": { 2024-04-21T10:41:23.983 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:23.983 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-21T10:41:23.983 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:23.983 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-21T10:41:23.983 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-21T10:41:23.983 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:23.983 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:23.983 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-21T10:41:23.983 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-21T10:41:23.984 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-21T10:41:23.984 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-21T10:41:23.984 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:23.984 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:23.984 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:23.984 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:23.984 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:23.984 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:23.984 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-21T10:41:24.687 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-21T10:41:24.687 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-21T10:41:24.687 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-21T10:41:24.688 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-21T10:41:24.688 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-21T10:41:24.688 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-21T10:41:24.688 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T10:41:24.688 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-21T10:41:24.688 DEBUG:tasks.check_counter:Getting stats from k 2024-04-21T10:41:24.688 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-21T10:41:24.929 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T10:41:25.686 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:25 smithi169 ceph-mon[31469]: pgmap v326: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:41:25.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:25 smithi190 ceph-mon[31423]: pgmap v326: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:41:25.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:25 smithi026 ceph-mon[27965]: pgmap v326: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-21T10:41:26.458 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-21T10:41:26.458 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T10:41:26.458 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 0, 2024-04-21T10:41:26.458 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 2, 2024-04-21T10:41:26.458 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 0, 2024-04-21T10:41:26.459 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 0, 2024-04-21T10:41:26.459 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-04-21T10:41:26.459 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 0, 2024-04-21T10:41:26.459 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.001277854, 2024-04-21T10:41:26.459 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T10:41:26.459 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.000348680, 2024-04-21T10:41:26.459 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T10:41:26.459 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:26.459 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:26.460 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:26.460 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:26.460 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:26.460 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:26.460 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:26.460 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:26.460 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:26.460 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:26.460 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T10:41:26.461 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T10:41:26.461 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:26.461 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T10:41:26.461 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 223, 2024-04-21T10:41:26.461 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 211, 2024-04-21T10:41:26.461 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1453408, 2024-04-21T10:41:26.461 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 128992, 2024-04-21T10:41:26.461 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 2, 2024-04-21T10:41:26.461 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 1, 2024-04-21T10:41:26.462 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.046180176, 2024-04-21T10:41:26.462 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.014021183, 2024-04-21T10:41:26.462 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.035061531, 2024-04-21T10:41:26.462 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.005403698, 2024-04-21T10:41:26.462 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:26.462 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 211, 2024-04-21T10:41:26.462 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.012777884, 2024-04-21T10:41:26.462 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000060558 2024-04-21T10:41:26.463 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:26.463 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:26.463 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:26.463 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:26.463 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:26.463 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:26.463 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1453408, 2024-04-21T10:41:26.463 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 128992 2024-04-21T10:41:26.463 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:26.464 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T10:41:26.464 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 2, 2024-04-21T10:41:26.464 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 160, 2024-04-21T10:41:26.464 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 320, 2024-04-21T10:41:26.464 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 184368, 2024-04-21T10:41:26.464 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-04-21T10:41:26.464 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-04-21T10:41:26.464 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.011227807, 2024-04-21T10:41:26.465 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.007180410, 2024-04-21T10:41:26.465 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.001780437, 2024-04-21T10:41:26.465 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T10:41:26.465 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:26.465 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 157, 2024-04-21T10:41:26.465 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.015920911, 2024-04-21T10:41:26.465 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000101407 2024-04-21T10:41:26.465 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:26.466 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:26.466 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:26.466 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:26.466 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:26.466 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:26.466 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T10:41:26.466 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 184368 2024-04-21T10:41:26.466 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:26.467 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-21T10:41:26.467 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 0, 2024-04-21T10:41:26.467 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-21T10:41:26.467 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:26.467 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-21T10:41:26.467 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-21T10:41:26.467 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-21T10:41:26.468 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T10:41:26.468 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-21T10:41:26.468 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T10:41:26.468 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-21T10:41:26.468 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T10:41:26.468 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T10:41:26.468 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T10:41:26.468 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T10:41:26.469 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T10:41:26.469 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-21T10:41:26.469 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T10:41:26.469 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T10:41:26.469 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-21T10:41:26.469 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-21T10:41:26.469 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-21T10:41:26.469 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-21T10:41:26.469 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T10:41:26.470 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T10:41:26.470 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T10:41:26.470 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T10:41:26.470 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T10:41:26.470 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-21T10:41:26.470 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-21T10:41:26.470 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-21T10:41:26.470 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T10:41:26.471 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T10:41:26.471 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-21T10:41:26.471 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-21T10:41:26.471 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-21T10:41:26.471 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-21T10:41:26.471 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T10:41:26.471 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-21T10:41:26.471 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T10:41:26.471 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-21T10:41:26.472 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 149007, 2024-04-21T10:41:26.472 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 30, 2024-04-21T10:41:26.472 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-21T10:41:26.472 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-21T10:41:26.472 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-21T10:41:26.472 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-21T10:41:26.472 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-21T10:41:26.472 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-21T10:41:26.472 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-21T10:41:26.473 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-21T10:41:26.473 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 0, 2024-04-21T10:41:26.473 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 0, 2024-04-21T10:41:26.473 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T10:41:26.473 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-21T10:41:26.473 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-21T10:41:26.473 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-21T10:41:26.473 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 928, 2024-04-21T10:41:26.473 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 2, 2024-04-21T10:41:26.474 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-21T10:41:26.474 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-21T10:41:26.474 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-21T10:41:26.474 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-21T10:41:26.474 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-21T10:41:26.474 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T10:41:26.474 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-21T10:41:26.474 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-21T10:41:26.474 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-21T10:41:26.475 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 221, 2024-04-21T10:41:26.475 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1419470, 2024-04-21T10:41:26.475 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:26.475 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 221, 2024-04-21T10:41:26.475 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-21T10:41:26.475 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-21T10:41:26.475 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 221, 2024-04-21T10:41:26.475 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1419470, 2024-04-21T10:41:26.476 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-21T10:41:26.476 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-21T10:41:26.476 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-21T10:41:26.476 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-21T10:41:26.476 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:26.476 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-21T10:41:26.476 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-21T10:41:27.124 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-21T10:41:27.124 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-21T10:41:27.124 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-21T10:41:27.124 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-21T10:41:27.124 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-21T10:41:27.124 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-21T10:41:27.125 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-21T10:41:27.125 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-21T10:41:27.125 DEBUG:tasks.check_counter:Getting stats from c 2024-04-21T10:41:27.125 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-21T10:41:27.385 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:41:27.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:27 smithi190 ceph-mon[31423]: pgmap v327: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T10:41:27.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:27 smithi026 ceph-mon[27965]: pgmap v327: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T10:41:27.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:27 smithi169 ceph-mon[31469]: pgmap v327: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-21T10:41:29.113 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-04-21T10:41:29.114 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T10:41:29.114 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 16373, 2024-04-21T10:41:29.114 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 33561, 2024-04-21T10:41:29.114 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 5107965, 2024-04-21T10:41:29.114 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 31348036, 2024-04-21T10:41:29.115 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 14, 2024-04-21T10:41:29.115 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-04-21T10:41:29.115 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 1.002636096, 2024-04-21T10:41:29.115 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.492255557, 2024-04-21T10:41:29.115 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.769767376, 2024-04-21T10:41:29.115 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.034863030, 2024-04-21T10:41:29.115 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:29.115 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 33559, 2024-04-21T10:41:29.116 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.070907939, 2024-04-21T10:41:29.116 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000031911 2024-04-21T10:41:29.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.116 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:29.116 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 44943, 2024-04-21T10:41:29.116 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.006221838, 2024-04-21T10:41:29.116 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000138 2024-04-21T10:41:29.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.117 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 4592, 2024-04-21T10:41:29.117 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 5728 2024-04-21T10:41:29.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.117 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T10:41:29.117 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1823, 2024-04-21T10:41:29.117 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 2267, 2024-04-21T10:41:29.117 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 1877254, 2024-04-21T10:41:29.117 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 13435460, 2024-04-21T10:41:29.118 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 12, 2024-04-21T10:41:29.118 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 8, 2024-04-21T10:41:29.118 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.209658296, 2024-04-21T10:41:29.118 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.095353494, 2024-04-21T10:41:29.118 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.079165234, 2024-04-21T10:41:29.118 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.037451152, 2024-04-21T10:41:29.118 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:29.118 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2267, 2024-04-21T10:41:29.119 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.134397477, 2024-04-21T10:41:29.119 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000059284 2024-04-21T10:41:29.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.119 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:29.119 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1332, 2024-04-21T10:41:29.119 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000214987, 2024-04-21T10:41:29.119 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000161 2024-04-21T10:41:29.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.120 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 1514944, 2024-04-21T10:41:29.120 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 451552 2024-04-21T10:41:29.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.120 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T10:41:29.120 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 5351, 2024-04-21T10:41:29.120 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 4446, 2024-04-21T10:41:29.120 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 1763867, 2024-04-21T10:41:29.120 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 7872858, 2024-04-21T10:41:29.121 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 11, 2024-04-21T10:41:29.121 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 7, 2024-04-21T10:41:29.121 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.338605585, 2024-04-21T10:41:29.121 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.152376723, 2024-04-21T10:41:29.121 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.180316675, 2024-04-21T10:41:29.121 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.032692226, 2024-04-21T10:41:29.121 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:29.121 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4443, 2024-04-21T10:41:29.122 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.158018523, 2024-04-21T10:41:29.122 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000035565 2024-04-21T10:41:29.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.122 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:29.122 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 7450, 2024-04-21T10:41:29.122 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001086291, 2024-04-21T10:41:29.122 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000145 2024-04-21T10:41:29.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.123 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-04-21T10:41:29.123 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 128768 2024-04-21T10:41:29.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.123 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-04-21T10:41:29.123 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-04-21T10:41:29.123 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-04-21T10:41:29.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.123 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-04-21T10:41:29.124 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-04-21T10:41:29.124 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-04-21T10:41:29.124 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 791, 2024-04-21T10:41:29.124 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 8.908489180, 2024-04-21T10:41:29.124 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.011262312 2024-04-21T10:41:29.124 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:29.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.124 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-04-21T10:41:29.124 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-04-21T10:41:29.125 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-04-21T10:41:29.125 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3690, 2024-04-21T10:41:29.125 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.487186189, 2024-04-21T10:41:29.125 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000132028 2024-04-21T10:41:29.125 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:29.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.125 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-04-21T10:41:29.126 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 8892, 2024-04-21T10:41:29.126 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 8768, 2024-04-21T10:41:29.126 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-04-21T10:41:29.126 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 8768, 2024-04-21T10:41:29.126 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 19.749055821, 2024-04-21T10:41:29.126 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002252401 2024-04-21T10:41:29.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.127 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-04-21T10:41:29.127 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 124, 2024-04-21T10:41:29.127 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-04-21T10:41:29.127 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-04-21T10:41:29.127 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 201, 2024-04-21T10:41:29.127 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 23, 2024-04-21T10:41:29.127 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-04-21T10:41:29.128 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 3903, 2024-04-21T10:41:29.128 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 246, 2024-04-21T10:41:29.128 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 3246, 2024-04-21T10:41:29.128 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 411, 2024-04-21T10:41:29.128 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 3551, 2024-04-21T10:41:29.128 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 1542, 2024-04-21T10:41:29.128 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 2301, 2024-04-21T10:41:29.128 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 2301, 2024-04-21T10:41:29.129 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 72, 2024-04-21T10:41:29.129 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 9176, 2024-04-21T10:41:29.129 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 8644, 2024-04-21T10:41:29.129 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 122, 2024-04-21T10:41:29.129 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 7, 2024-04-21T10:41:29.129 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-04-21T10:41:29.129 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-04-21T10:41:29.129 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-04-21T10:41:29.130 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2024-04-21T10:41:29.130 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-04-21T10:41:29.130 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 69, 2024-04-21T10:41:29.130 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 1533, 2024-04-21T10:41:29.130 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 3, 2024-04-21T10:41:29.130 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 327, 2024-04-21T10:41:29.130 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-04-21T10:41:29.130 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2024-04-21T10:41:29.131 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-04-21T10:41:29.131 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 4, 2024-04-21T10:41:29.131 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 234, 2024-04-21T10:41:29.131 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2024-04-21T10:41:29.131 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T10:41:29.131 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-04-21T10:41:29.131 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T10:41:29.131 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-04-21T10:41:29.132 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-04-21T10:41:29.132 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T10:41:29.132 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T10:41:29.132 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2024-04-21T10:41:29.132 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 10, 2024-04-21T10:41:29.132 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 74, 2024-04-21T10:41:29.132 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 9146, 2024-04-21T10:41:29.132 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 8, 2024-04-21T10:41:29.132 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T10:41:29.133 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T10:41:29.133 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 5672, 2024-04-21T10:41:29.133 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 4414, 2024-04-21T10:41:29.133 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 760, 2024-04-21T10:41:29.133 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 2532 2024-04-21T10:41:29.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.133 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-04-21T10:41:29.133 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 37, 2024-04-21T10:41:29.134 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 18, 2024-04-21T10:41:29.134 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 12, 2024-04-21T10:41:29.134 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 14, 2024-04-21T10:41:29.134 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 269, 2024-04-21T10:41:29.134 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-04-21T10:41:29.134 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-04-21T10:41:29.134 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-04-21T10:41:29.135 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 2281, 2024-04-21T10:41:29.135 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 2281, 2024-04-21T10:41:29.135 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-04-21T10:41:29.135 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-04-21T10:41:29.135 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-04-21T10:41:29.135 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-04-21T10:41:29.135 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-04-21T10:41:29.135 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-04-21T10:41:29.136 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-04-21T10:41:29.136 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 2, 2024-04-21T10:41:29.136 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 11, 2024-04-21T10:41:29.136 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T10:41:29.136 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 69, 2024-04-21T10:41:29.136 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-04-21T10:41:29.136 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 23, 2024-04-21T10:41:29.136 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-04-21T10:41:29.137 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2024-04-21T10:41:29.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.137 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-04-21T10:41:29.137 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-04-21T10:41:29.137 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 17567, 2024-04-21T10:41:29.137 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-04-21T10:41:29.137 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-04-21T10:41:29.137 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 17567, 2024-04-21T10:41:29.138 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-04-21T10:41:29.138 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-04-21T10:41:29.138 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 24, 2024-04-21T10:41:29.138 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-04-21T10:41:29.138 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-04-21T10:41:29.138 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 24, 2024-04-21T10:41:29.138 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 3, 2024-04-21T10:41:29.138 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-04-21T10:41:29.139 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-04-21T10:41:29.139 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2024-04-21T10:41:29.139 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 42080911, 2024-04-21T10:41:29.139 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2024-04-21T10:41:29.139 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-04-21T10:41:29.139 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 391, 2024-04-21T10:41:29.139 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 8.895393799, 2024-04-21T10:41:29.139 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.022750367 2024-04-21T10:41:29.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.140 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2024-04-21T10:41:29.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.140 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-04-21T10:41:29.140 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 2327, 2024-04-21T10:41:29.140 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 6133, 2024-04-21T10:41:29.140 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 3806, 2024-04-21T10:41:29.140 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 246, 2024-04-21T10:41:29.140 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 281, 2024-04-21T10:41:29.141 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 35, 2024-04-21T10:41:29.141 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 3903, 2024-04-21T10:41:29.141 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 7480, 2024-04-21T10:41:29.141 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 3577, 2024-04-21T10:41:29.141 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 2301, 2024-04-21T10:41:29.141 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 6117, 2024-04-21T10:41:29.141 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 3816, 2024-04-21T10:41:29.141 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 90904, 2024-04-21T10:41:29.141 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 223608 2024-04-21T10:41:29.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.142 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-04-21T10:41:29.142 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 9220, 2024-04-21T10:41:29.142 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 1602, 2024-04-21T10:41:29.142 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 8892, 2024-04-21T10:41:29.142 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 94, 2024-04-21T10:41:29.142 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 3226, 2024-04-21T10:41:29.142 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-04-21T10:41:29.142 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5999, 2024-04-21T10:41:29.143 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 12.903428766, 2024-04-21T10:41:29.143 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002150929 2024-04-21T10:41:29.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.143 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-04-21T10:41:29.143 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 19, 2024-04-21T10:41:29.143 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.097861117, 2024-04-21T10:41:29.143 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.005150585 2024-04-21T10:41:29.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.143 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-04-21T10:41:29.144 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.144 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.144 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.144 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-04-21T10:41:29.144 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.144 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.144 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.144 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-04-21T10:41:29.145 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 83, 2024-04-21T10:41:29.145 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.017760541, 2024-04-21T10:41:29.145 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000213982 2024-04-21T10:41:29.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.145 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-04-21T10:41:29.145 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.145 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.145 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.146 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-04-21T10:41:29.146 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2024-04-21T10:41:29.146 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000138923, 2024-04-21T10:41:29.146 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000138923 2024-04-21T10:41:29.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.146 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-04-21T10:41:29.146 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.146 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.147 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.147 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-04-21T10:41:29.147 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.147 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.147 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.147 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-04-21T10:41:29.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.148 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.148 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.148 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-04-21T10:41:29.149 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.149 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.149 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.149 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-04-21T10:41:29.149 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 59, 2024-04-21T10:41:29.149 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.048981208, 2024-04-21T10:41:29.149 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000830189 2024-04-21T10:41:29.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.150 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-04-21T10:41:29.150 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.150 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.150 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.151 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-04-21T10:41:29.151 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.151 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.151 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.151 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-04-21T10:41:29.151 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 169, 2024-04-21T10:41:29.151 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.046950699, 2024-04-21T10:41:29.151 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000277814 2024-04-21T10:41:29.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.152 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-04-21T10:41:29.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 101, 2024-04-21T10:41:29.152 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.617799661, 2024-04-21T10:41:29.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.006116828 2024-04-21T10:41:29.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.152 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-04-21T10:41:29.152 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 62, 2024-04-21T10:41:29.152 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.631937655, 2024-04-21T10:41:29.153 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.010192542 2024-04-21T10:41:29.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.153 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-04-21T10:41:29.153 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.153 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.153 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.153 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-04-21T10:41:29.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.154 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.154 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-04-21T10:41:29.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 22, 2024-04-21T10:41:29.154 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.640885979, 2024-04-21T10:41:29.154 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.029131180 2024-04-21T10:41:29.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.155 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-04-21T10:41:29.155 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.155 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.155 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.155 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-04-21T10:41:29.155 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.155 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.156 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.156 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-04-21T10:41:29.156 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 8, 2024-04-21T10:41:29.156 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.050523709, 2024-04-21T10:41:29.156 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.006315463 2024-04-21T10:41:29.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.156 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-04-21T10:41:29.156 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.157 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.157 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-04-21T10:41:29.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 8, 2024-04-21T10:41:29.157 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.002801001, 2024-04-21T10:41:29.157 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000350125 2024-04-21T10:41:29.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.158 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-04-21T10:41:29.158 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.158 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.158 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.158 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-04-21T10:41:29.158 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.158 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.159 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.159 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-04-21T10:41:29.159 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 6, 2024-04-21T10:41:29.159 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.004802634, 2024-04-21T10:41:29.159 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000800439 2024-04-21T10:41:29.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.159 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-04-21T10:41:29.159 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2231, 2024-04-21T10:41:29.160 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 4.685183928, 2024-04-21T10:41:29.160 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002100037 2024-04-21T10:41:29.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.160 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-04-21T10:41:29.160 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-04-21T10:41:29.160 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-04-21T10:41:29.160 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.160 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.160 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.161 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:29.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.161 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-04-21T10:41:29.161 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2024-04-21T10:41:29.161 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 5, 2024-04-21T10:41:29.161 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 4, 2024-04-21T10:41:29.161 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2024-04-21T10:41:29.161 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-04-21T10:41:29.161 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 6788, 2024-04-21T10:41:29.162 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 6788, 2024-04-21T10:41:29.162 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 620, 2024-04-21T10:41:29.162 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-04-21T10:41:29.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.162 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-04-21T10:41:29.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-04-21T10:41:29.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-04-21T10:41:29.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T10:41:29.162 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-04-21T10:41:29.163 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T10:41:29.163 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-04-21T10:41:29.163 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T10:41:29.163 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T10:41:29.163 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T10:41:29.163 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T10:41:29.163 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T10:41:29.163 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-04-21T10:41:29.164 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T10:41:29.164 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T10:41:29.164 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-04-21T10:41:29.164 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-04-21T10:41:29.164 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-04-21T10:41:29.164 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-04-21T10:41:29.164 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T10:41:29.164 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T10:41:29.164 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T10:41:29.165 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T10:41:29.165 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T10:41:29.165 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-04-21T10:41:29.165 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-04-21T10:41:29.165 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-04-21T10:41:29.165 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T10:41:29.165 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T10:41:29.165 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-04-21T10:41:29.165 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-04-21T10:41:29.166 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-04-21T10:41:29.166 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-04-21T10:41:29.166 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T10:41:29.166 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-04-21T10:41:29.166 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T10:41:29.166 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-04-21T10:41:29.166 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 390558, 2024-04-21T10:41:29.167 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 2428, 2024-04-21T10:41:29.167 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 440, 2024-04-21T10:41:29.167 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 5, 2024-04-21T10:41:29.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-04-21T10:41:29.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-04-21T10:41:29.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-04-21T10:41:29.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-04-21T10:41:29.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-04-21T10:41:29.167 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-04-21T10:41:29.168 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 8136, 2024-04-21T10:41:29.168 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 170, 2024-04-21T10:41:29.168 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T10:41:29.168 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-04-21T10:41:29.168 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-04-21T10:41:29.168 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-04-21T10:41:29.168 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 9529995, 2024-04-21T10:41:29.168 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 109281, 2024-04-21T10:41:29.169 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-04-21T10:41:29.169 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-04-21T10:41:29.169 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-04-21T10:41:29.169 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-04-21T10:41:29.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.169 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-04-21T10:41:29.169 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-04-21T10:41:29.169 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-04-21T10:41:29.169 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 3082, 2024-04-21T10:41:29.170 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 41222838, 2024-04-21T10:41:29.170 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 33, 2024-04-21T10:41:29.170 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 3082, 2024-04-21T10:41:29.170 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-04-21T10:41:29.170 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3082, 2024-04-21T10:41:29.170 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 9.817721165, 2024-04-21T10:41:29.170 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.003185503 2024-04-21T10:41:29.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.171 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-04-21T10:41:29.171 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-04-21T10:41:29.171 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3082, 2024-04-21T10:41:29.171 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 3357 2024-04-21T10:41:29.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.171 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 3082, 2024-04-21T10:41:29.171 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 34, 2024-04-21T10:41:29.171 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 3048, 2024-04-21T10:41:29.172 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-04-21T10:41:29.172 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-04-21T10:41:29.172 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2024-04-21T10:41:29.172 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2024-04-21T10:41:29.172 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 34, 2024-04-21T10:41:29.172 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 425, 2024-04-21T10:41:29.172 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 32, 2024-04-21T10:41:29.172 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-04-21T10:41:29.173 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-04-21T10:41:29.173 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-04-21T10:41:29.173 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 7, 2024-04-21T10:41:29.173 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 2318, 2024-04-21T10:41:29.173 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-04-21T10:41:29.173 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-04-21T10:41:29.173 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-04-21T10:41:29.173 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 0, 2024-04-21T10:41:29.173 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2024-04-21T10:41:29.174 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-04-21T10:41:29.174 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-04-21T10:41:29.174 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-04-21T10:41:29.174 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-04-21T10:41:29.174 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-04-21T10:41:29.174 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-04-21T10:41:29.174 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T10:41:29.175 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-04-21T10:41:29.175 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-04-21T10:41:29.175 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 8, 2024-04-21T10:41:29.175 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-04-21T10:41:29.175 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-04-21T10:41:29.175 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-04-21T10:41:29.175 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-04-21T10:41:29.175 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-04-21T10:41:29.175 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-04-21T10:41:29.176 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-04-21T10:41:29.176 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-04-21T10:41:29.176 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-04-21T10:41:29.176 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-04-21T10:41:29.176 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-04-21T10:41:29.176 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-04-21T10:41:29.176 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-04-21T10:41:29.176 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-04-21T10:41:29.176 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-04-21T10:41:29.177 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-04-21T10:41:29.177 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 100, 2024-04-21T10:41:29.177 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-04-21T10:41:29.177 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 13, 2024-04-21T10:41:29.177 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2024-04-21T10:41:29.177 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2024-04-21T10:41:29.177 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-04-21T10:41:29.177 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-04-21T10:41:29.177 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 474, 2024-04-21T10:41:29.178 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-04-21T10:41:29.178 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 29 2024-04-21T10:41:29.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.178 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-04-21T10:41:29.178 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2024-04-21T10:41:29.178 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-04-21T10:41:29.178 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-04-21T10:41:29.178 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-04-21T10:41:29.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.179 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2024-04-21T10:41:29.179 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 12 2024-04-21T10:41:29.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.179 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-04-21T10:41:29.179 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-04-21T10:41:29.179 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 10, 2024-04-21T10:41:29.179 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-04-21T10:41:29.180 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 10, 2024-04-21T10:41:29.180 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 2281, 2024-04-21T10:41:29.180 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 2281, 2024-04-21T10:41:29.180 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-04-21T10:41:29.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.180 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T10:41:29.180 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:29.180 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-04-21T10:41:29.180 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:29.181 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 23543, 2024-04-21T10:41:29.181 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 6612237, 2024-04-21T10:41:29.181 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:29.181 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 23543, 2024-04-21T10:41:29.181 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-04-21T10:41:29.181 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-04-21T10:41:29.181 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 23543, 2024-04-21T10:41:29.181 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 6612237, 2024-04-21T10:41:29.181 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:29.182 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.182 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.182 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.182 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:29.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.182 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-04-21T10:41:29.182 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:29.182 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-04-21T10:41:29.183 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:29.183 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-04-21T10:41:29.183 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-04-21T10:41:29.183 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:29.183 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:29.183 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3082, 2024-04-21T10:41:29.183 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 41453219, 2024-04-21T10:41:29.183 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3082, 2024-04-21T10:41:29.184 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 41453219, 2024-04-21T10:41:29.184 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:29.184 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.184 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.184 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.184 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:29.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.184 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-04-21T10:41:29.184 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:29.185 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-04-21T10:41:29.185 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:29.185 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-04-21T10:41:29.185 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-04-21T10:41:29.185 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:29.185 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:29.185 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 3082, 2024-04-21T10:41:29.185 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 3082, 2024-04-21T10:41:29.185 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 3082, 2024-04-21T10:41:29.185 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 3082, 2024-04-21T10:41:29.186 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:29.186 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.186 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.186 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.186 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:29.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.186 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-04-21T10:41:29.186 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:29.187 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-04-21T10:41:29.187 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:29.187 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 2281, 2024-04-21T10:41:29.187 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 230381, 2024-04-21T10:41:29.187 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:29.187 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 2281, 2024-04-21T10:41:29.187 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-04-21T10:41:29.187 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-04-21T10:41:29.188 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 34, 2024-04-21T10:41:29.188 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 230381, 2024-04-21T10:41:29.188 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:29.188 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.188 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.188 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.188 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:29.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:29.188 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55c003840500": { 2024-04-21T10:41:29.189 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:29.189 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-04-21T10:41:29.189 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:29.189 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 17567, 2024-04-21T10:41:29.189 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 37886607, 2024-04-21T10:41:29.189 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:29.189 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 17567, 2024-04-21T10:41:29.189 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-04-21T10:41:29.189 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-04-21T10:41:29.190 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 391, 2024-04-21T10:41:29.190 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 37886607, 2024-04-21T10:41:29.190 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:29.190 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:29.190 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:29.190 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:29.190 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:29.190 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:29.191 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-04-21T10:41:29.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:29 smithi190 ceph-mon[31423]: pgmap v328: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:41:29.889 INFO:tasks.check_counter:Daemon mds.c mds.exported=69 2024-04-21T10:41:29.889 INFO:tasks.check_counter:Daemon mds.c mds.imported=3 2024-04-21T10:41:29.889 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=37 2024-04-21T10:41:29.889 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=18 2024-04-21T10:41:29.889 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-21T10:41:29.890 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-21T10:41:29.890 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T10:41:29.890 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=23 2024-04-21T10:41:29.890 DEBUG:tasks.check_counter:Getting stats from f 2024-04-21T10:41:29.890 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-21T10:41:29.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:29 smithi026 ceph-mon[27965]: pgmap v328: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:41:29.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:29 smithi169 ceph-mon[31469]: pgmap v328: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:41:30.132 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:41:31.789 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:31 smithi190 ceph-mon[31423]: pgmap v329: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:41:31.789 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-04-21T10:41:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T10:41:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1232, 2024-04-21T10:41:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1215, 2024-04-21T10:41:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 6010476, 2024-04-21T10:41:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 445372, 2024-04-21T10:41:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2024-04-21T10:41:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 1, 2024-04-21T10:41:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.160002346, 2024-04-21T10:41:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.048624634, 2024-04-21T10:41:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.066485410, 2024-04-21T10:41:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.042518076, 2024-04-21T10:41:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1214, 2024-04-21T10:41:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.059517196, 2024-04-21T10:41:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000049025 2024-04-21T10:41:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 1502176, 2024-04-21T10:41:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 131184 2024-04-21T10:41:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T10:41:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1135, 2024-04-21T10:41:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1292, 2024-04-21T10:41:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 2302123, 2024-04-21T10:41:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 802280, 2024-04-21T10:41:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 7, 2024-04-21T10:41:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 5, 2024-04-21T10:41:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.094018414, 2024-04-21T10:41:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.035793348, 2024-04-21T10:41:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.021621155, 2024-04-21T10:41:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.028449924, 2024-04-21T10:41:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1289, 2024-04-21T10:41:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.069956107, 2024-04-21T10:41:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000054271 2024-04-21T10:41:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T10:41:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 441920 2024-04-21T10:41:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T10:41:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 207, 2024-04-21T10:41:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 207, 2024-04-21T10:41:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 4253501, 2024-04-21T10:41:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 65009, 2024-04-21T10:41:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 4, 2024-04-21T10:41:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-04-21T10:41:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.021477362, 2024-04-21T10:41:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.006934141, 2024-04-21T10:41:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.006445726, 2024-04-21T10:41:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.006321507, 2024-04-21T10:41:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 206, 2024-04-21T10:41:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.008816182, 2024-04-21T10:41:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000042797 2024-04-21T10:41:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-21T10:41:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T10:41:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-04-21T10:41:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-04-21T10:41:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-04-21T10:41:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-04-21T10:41:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-04-21T10:41:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-04-21T10:41:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3876, 2024-04-21T10:41:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.133022351, 2024-04-21T10:41:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000034319 2024-04-21T10:41:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-04-21T10:41:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-04-21T10:41:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-04-21T10:41:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-04-21T10:41:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 0, 2024-04-21T10:41:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 0, 2024-04-21T10:41:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-04-21T10:41:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-04-21T10:41:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2024-04-21T10:41:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-04-21T10:41:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-04-21T10:41:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 0, 2024-04-21T10:41:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-04-21T10:41:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-04-21T10:41:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 2497, 2024-04-21T10:41:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 1582, 2024-04-21T10:41:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 915, 2024-04-21T10:41:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 0, 2024-04-21T10:41:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 659, 2024-04-21T10:41:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 0, 2024-04-21T10:41:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2024-04-21T10:41:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2024-04-21T10:41:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 69, 2024-04-21T10:41:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 0, 2024-04-21T10:41:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 0, 2024-04-21T10:41:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2024-04-21T10:41:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 0, 2024-04-21T10:41:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-04-21T10:41:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-04-21T10:41:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2024-04-21T10:41:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2024-04-21T10:41:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-04-21T10:41:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2024-04-21T10:41:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2024-04-21T10:41:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2024-04-21T10:41:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2024-04-21T10:41:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-04-21T10:41:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2024-04-21T10:41:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-04-21T10:41:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 3453, 2024-04-21T10:41:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 26601754, 2024-04-21T10:41:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 6, 2024-04-21T10:41:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T10:41:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-04-21T10:41:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T10:41:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-04-21T10:41:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-04-21T10:41:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T10:41:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T10:41:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2024-04-21T10:41:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-04-21T10:41:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T10:41:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2024-04-21T10:41:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T10:41:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T10:41:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T10:41:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2024-04-21T10:41:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2024-04-21T10:41:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2024-04-21T10:41:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2024-04-21T10:41:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-04-21T10:41:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2024-04-21T10:41:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 0, 2024-04-21T10:41:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 0, 2024-04-21T10:41:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 0, 2024-04-21T10:41:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2024-04-21T10:41:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-04-21T10:41:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-04-21T10:41:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-04-21T10:41:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2024-04-21T10:41:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2024-04-21T10:41:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2024-04-21T10:41:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-04-21T10:41:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-04-21T10:41:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-04-21T10:41:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-04-21T10:41:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-04-21T10:41:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-04-21T10:41:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 0, 2024-04-21T10:41:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 0, 2024-04-21T10:41:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T10:41:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2024-04-21T10:41:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-04-21T10:41:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-04-21T10:41:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-04-21T10:41:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2024-04-21T10:41:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-04-21T10:41:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-04-21T10:41:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 0, 2024-04-21T10:41:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2024-04-21T10:41:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2024-04-21T10:41:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 6494, 2024-04-21T10:41:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-04-21T10:41:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2024-04-21T10:41:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 0, 2024-04-21T10:41:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2024-04-21T10:41:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2024-04-21T10:41:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 2, 2024-04-21T10:41:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 10, 2024-04-21T10:41:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-04-21T10:41:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2024-04-21T10:41:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 13576516, 2024-04-21T10:41:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 14596701, 2024-04-21T10:41:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 14596589, 2024-04-21T10:41:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-04-21T10:41:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 6494 2024-04-21T10:41:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-04-21T10:41:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 1233, 2024-04-21T10:41:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 2242, 2024-04-21T10:41:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 1009, 2024-04-21T10:41:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 89, 2024-04-21T10:41:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 111, 2024-04-21T10:41:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 22, 2024-04-21T10:41:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 2497, 2024-04-21T10:41:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 2574, 2024-04-21T10:41:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 77, 2024-04-21T10:41:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2024-04-21T10:41:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 0, 2024-04-21T10:41:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2024-04-21T10:41:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 61568, 2024-04-21T10:41:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 215416 2024-04-21T10:41:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-04-21T10:41:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 0, 2024-04-21T10:41:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2024-04-21T10:41:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 0, 2024-04-21T10:41:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 0, 2024-04-21T10:41:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 0, 2024-04-21T10:41:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-04-21T10:41:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-04-21T10:41:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-04-21T10:41:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-04-21T10:41:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-04-21T10:41:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-04-21T10:41:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-04-21T10:41:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-04-21T10:41:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-04-21T10:41:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-04-21T10:41:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-04-21T10:41:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-04-21T10:41:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-04-21T10:41:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-04-21T10:41:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-04-21T10:41:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-04-21T10:41:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-04-21T10:41:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-04-21T10:41:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-04-21T10:41:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-04-21T10:41:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-04-21T10:41:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-04-21T10:41:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-04-21T10:41:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-04-21T10:41:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-04-21T10:41:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-04-21T10:41:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-04-21T10:41:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-04-21T10:41:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-04-21T10:41:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-04-21T10:41:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-04-21T10:41:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-04-21T10:41:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-04-21T10:41:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 2, 2024-04-21T10:41:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 6, 2024-04-21T10:41:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 4, 2024-04-21T10:41:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 5, 2024-04-21T10:41:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-04-21T10:41:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 0, 2024-04-21T10:41:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 0, 2024-04-21T10:41:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 3441, 2024-04-21T10:41:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-04-21T10:41:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-04-21T10:41:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-04-21T10:41:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-04-21T10:41:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T10:41:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-04-21T10:41:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T10:41:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-04-21T10:41:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T10:41:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T10:41:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T10:41:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T10:41:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T10:41:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-04-21T10:41:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T10:41:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T10:41:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-04-21T10:41:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-04-21T10:41:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-04-21T10:41:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-04-21T10:41:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T10:41:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T10:41:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T10:41:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T10:41:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T10:41:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-04-21T10:41:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-04-21T10:41:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-04-21T10:41:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T10:41:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T10:41:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-04-21T10:41:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-04-21T10:41:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-04-21T10:41:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-04-21T10:41:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T10:41:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-04-21T10:41:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T10:41:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-04-21T10:41:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 234489, 2024-04-21T10:41:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 1269, 2024-04-21T10:41:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-04-21T10:41:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-04-21T10:41:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-04-21T10:41:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-04-21T10:41:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-04-21T10:41:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-04-21T10:41:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-04-21T10:41:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-04-21T10:41:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 8136, 2024-04-21T10:41:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 170, 2024-04-21T10:41:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T10:41:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-04-21T10:41:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-04-21T10:41:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-04-21T10:41:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 4713095, 2024-04-21T10:41:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 38738, 2024-04-21T10:41:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-04-21T10:41:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-04-21T10:41:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-04-21T10:41:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-04-21T10:41:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-04-21T10:41:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-04-21T10:41:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-04-21T10:41:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 2324, 2024-04-21T10:41:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 26, 2024-04-21T10:41:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 4, 2024-04-21T10:41:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 2324, 2024-04-21T10:41:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-04-21T10:41:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2324, 2024-04-21T10:41:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.195965910, 2024-04-21T10:41:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000514615 2024-04-21T10:41:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-04-21T10:41:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-04-21T10:41:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2324, 2024-04-21T10:41:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 2325 2024-04-21T10:41:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 2324, 2024-04-21T10:41:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 2324, 2024-04-21T10:41:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2024-04-21T10:41:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-04-21T10:41:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-04-21T10:41:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 1481, 2024-04-21T10:41:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2024-04-21T10:41:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 840, 2024-04-21T10:41:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2024-04-21T10:41:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2024-04-21T10:41:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-04-21T10:41:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-04-21T10:41:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-04-21T10:41:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-04-21T10:41:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2024-04-21T10:41:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-04-21T10:41:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-04-21T10:41:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-04-21T10:41:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 2, 2024-04-21T10:41:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2024-04-21T10:41:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-04-21T10:41:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-04-21T10:41:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-04-21T10:41:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-04-21T10:41:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-04-21T10:41:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-04-21T10:41:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T10:41:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-04-21T10:41:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-04-21T10:41:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-04-21T10:41:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-04-21T10:41:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-04-21T10:41:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-04-21T10:41:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-04-21T10:41:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-04-21T10:41:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-04-21T10:41:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-04-21T10:41:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-04-21T10:41:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-04-21T10:41:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-04-21T10:41:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-04-21T10:41:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-04-21T10:41:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-04-21T10:41:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-04-21T10:41:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-04-21T10:41:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-04-21T10:41:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 100, 2024-04-21T10:41:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-04-21T10:41:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 13, 2024-04-21T10:41:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 5, 2024-04-21T10:41:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 5, 2024-04-21T10:41:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-04-21T10:41:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-04-21T10:41:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2024-04-21T10:41:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 2, 2024-04-21T10:41:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2024-04-21T10:41:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-04-21T10:41:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 0, 2024-04-21T10:41:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-04-21T10:41:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-04-21T10:41:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-04-21T10:41:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2024-04-21T10:41:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2024-04-21T10:41:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-04-21T10:41:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-04-21T10:41:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T10:41:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-04-21T10:41:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2024-04-21T10:41:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 0, 2024-04-21T10:41:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2024-04-21T10:41:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-04-21T10:41:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T10:41:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-04-21T10:41:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 2570, 2024-04-21T10:41:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 12319257, 2024-04-21T10:41:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 2570, 2024-04-21T10:41:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-04-21T10:41:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-04-21T10:41:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2570, 2024-04-21T10:41:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 12319257, 2024-04-21T10:41:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-04-21T10:41:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-04-21T10:41:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-04-21T10:41:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-04-21T10:41:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 2324, 2024-04-21T10:41:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 10408392, 2024-04-21T10:41:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2324, 2024-04-21T10:41:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 10408392, 2024-04-21T10:41:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-04-21T10:41:31.863 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:31.863 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-04-21T10:41:31.863 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:31.863 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-04-21T10:41:31.863 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-04-21T10:41:31.863 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:31.863 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:31.863 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 2324, 2024-04-21T10:41:31.863 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 2324, 2024-04-21T10:41:31.864 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2324, 2024-04-21T10:41:31.864 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2324, 2024-04-21T10:41:31.864 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:31.864 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.864 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.864 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.864 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:31.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.865 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-04-21T10:41:31.865 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:31.865 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-04-21T10:41:31.865 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:31.865 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-04-21T10:41:31.865 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-04-21T10:41:31.865 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:31.865 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:31.865 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-04-21T10:41:31.866 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-04-21T10:41:31.866 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-04-21T10:41:31.866 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-04-21T10:41:31.866 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:31.866 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.866 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.866 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.866 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:31.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:31.867 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x563f5b35a6e0": { 2024-04-21T10:41:31.867 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:31.867 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-04-21T10:41:31.867 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:31.867 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-04-21T10:41:31.867 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-04-21T10:41:31.867 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:31.868 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:31.868 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-04-21T10:41:31.868 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-04-21T10:41:31.868 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2024-04-21T10:41:31.868 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2024-04-21T10:41:31.868 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:31.868 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:31.868 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:31.868 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:31.869 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:31.869 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:31.869 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-04-21T10:41:31.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:31 smithi026 ceph-mon[27965]: pgmap v329: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:41:31.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:31 smithi169 ceph-mon[31469]: pgmap v329: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:41:32.597 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-21T10:41:32.598 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-21T10:41:32.598 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-21T10:41:32.598 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-21T10:41:32.598 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-21T10:41:32.598 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-21T10:41:32.598 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T10:41:32.598 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-21T10:41:32.598 DEBUG:tasks.check_counter:Getting stats from i 2024-04-21T10:41:32.599 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-21T10:41:32.838 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:41:33.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:33 smithi026 ceph-mon[27965]: pgmap v330: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:41:33.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:33 smithi169 ceph-mon[31469]: pgmap v330: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:41:34.004 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:33 smithi190 ceph-mon[31423]: pgmap v330: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:41:34.599 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-04-21T10:41:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T10:41:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 3273, 2024-04-21T10:41:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 2666, 2024-04-21T10:41:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 1247559, 2024-04-21T10:41:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 9027320, 2024-04-21T10:41:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 11, 2024-04-21T10:41:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2024-04-21T10:41:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.322655133, 2024-04-21T10:41:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.103715595, 2024-04-21T10:41:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.093197669, 2024-04-21T10:41:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.112935068, 2024-04-21T10:41:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2668, 2024-04-21T10:41:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.955621627, 2024-04-21T10:41:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000358179 2024-04-21T10:41:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2611, 2024-04-21T10:41:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000510602, 2024-04-21T10:41:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000195 2024-04-21T10:41:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 380640, 2024-04-21T10:41:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 101456 2024-04-21T10:41:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T10:41:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 5149, 2024-04-21T10:41:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 6153, 2024-04-21T10:41:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 3328312, 2024-04-21T10:41:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 5442170, 2024-04-21T10:41:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 39, 2024-04-21T10:41:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 7, 2024-04-21T10:41:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.414580553, 2024-04-21T10:41:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.181175453, 2024-04-21T10:41:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.177773508, 2024-04-21T10:41:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.062498820, 2024-04-21T10:41:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 6153, 2024-04-21T10:41:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 2.684012974, 2024-04-21T10:41:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000436212 2024-04-21T10:41:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 7581, 2024-04-21T10:41:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001429576, 2024-04-21T10:41:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000188 2024-04-21T10:41:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 44256, 2024-04-21T10:41:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 451584 2024-04-21T10:41:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T10:41:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 9438, 2024-04-21T10:41:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 13209, 2024-04-21T10:41:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 4453619, 2024-04-21T10:41:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 8097581, 2024-04-21T10:41:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 15, 2024-04-21T10:41:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 8, 2024-04-21T10:41:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.593983071, 2024-04-21T10:41:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.224388289, 2024-04-21T10:41:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.420811497, 2024-04-21T10:41:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.091483590, 2024-04-21T10:41:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 13202, 2024-04-21T10:41:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.394250567, 2024-04-21T10:41:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000105609 2024-04-21T10:41:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 18234, 2024-04-21T10:41:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.002568473, 2024-04-21T10:41:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000140 2024-04-21T10:41:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 1621296, 2024-04-21T10:41:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 174336 2024-04-21T10:41:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-04-21T10:41:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2024-04-21T10:41:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-04-21T10:41:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2024-04-21T10:41:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-04-21T10:41:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-04-21T10:41:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 645, 2024-04-21T10:41:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 3.022183139, 2024-04-21T10:41:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.004685555 2024-04-21T10:41:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2024-04-21T10:41:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2024-04-21T10:41:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2024-04-21T10:41:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1467, 2024-04-21T10:41:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.245941168, 2024-04-21T10:41:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000167649 2024-04-21T10:41:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2024-04-21T10:41:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 4655, 2024-04-21T10:41:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 4598, 2024-04-21T10:41:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2024-04-21T10:41:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4598, 2024-04-21T10:41:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 16.810957155, 2024-04-21T10:41:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.003656145 2024-04-21T10:41:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2024-04-21T10:41:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 57, 2024-04-21T10:41:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2024-04-21T10:41:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2024-04-21T10:41:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 269, 2024-04-21T10:41:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2024-04-21T10:41:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2024-04-21T10:41:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 1560, 2024-04-21T10:41:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 230, 2024-04-21T10:41:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 1317, 2024-04-21T10:41:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 13, 2024-04-21T10:41:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 670, 2024-04-21T10:41:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 336, 2024-04-21T10:41:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 15, 2024-04-21T10:41:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 15, 2024-04-21T10:41:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 70, 2024-04-21T10:41:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 8964, 2024-04-21T10:41:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 6883, 2024-04-21T10:41:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 54, 2024-04-21T10:41:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 79, 2024-04-21T10:41:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2024-04-21T10:41:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2024-04-21T10:41:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 13, 2024-04-21T10:41:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2024-04-21T10:41:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2024-04-21T10:41:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 3, 2024-04-21T10:41:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 327, 2024-04-21T10:41:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 67, 2024-04-21T10:41:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 1517, 2024-04-21T10:41:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2024-04-21T10:41:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2024-04-21T10:41:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2024-04-21T10:41:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 3453, 2024-04-21T10:41:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 26601754, 2024-04-21T10:41:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 6, 2024-04-21T10:41:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T10:41:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2024-04-21T10:41:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T10:41:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2024-04-21T10:41:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2024-04-21T10:41:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T10:41:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T10:41:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2024-04-21T10:41:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2024-04-21T10:41:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 32, 2024-04-21T10:41:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 1691, 2024-04-21T10:41:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T10:41:34.618 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T10:41:34.618 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T10:41:34.618 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 2526, 2024-04-21T10:41:34.618 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 2164, 2024-04-21T10:41:34.618 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 44, 2024-04-21T10:41:34.618 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 2000 2024-04-21T10:41:34.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.618 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2024-04-21T10:41:34.619 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 208, 2024-04-21T10:41:34.619 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 48, 2024-04-21T10:41:34.619 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 30, 2024-04-21T10:41:34.619 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 35, 2024-04-21T10:41:34.619 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 225, 2024-04-21T10:41:34.619 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2024-04-21T10:41:34.619 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2024-04-21T10:41:34.619 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2024-04-21T10:41:34.619 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 962, 2024-04-21T10:41:34.620 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 959, 2024-04-21T10:41:34.620 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 3, 2024-04-21T10:41:34.620 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2024-04-21T10:41:34.620 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2024-04-21T10:41:34.620 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2024-04-21T10:41:34.620 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2024-04-21T10:41:34.620 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2024-04-21T10:41:34.620 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2024-04-21T10:41:34.621 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_path": 2, 2024-04-21T10:41:34.621 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_quiesce_inode": 24, 2024-04-21T10:41:34.621 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T10:41:34.621 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 4, 2024-04-21T10:41:34.621 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2024-04-21T10:41:34.621 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2024-04-21T10:41:34.621 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2024-04-21T10:41:34.621 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2024-04-21T10:41:34.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.622 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2024-04-21T10:41:34.622 INFO:teuthology.orchestra.run.smithi190.stdout: "evlrg": 0, 2024-04-21T10:41:34.622 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 6494, 2024-04-21T10:41:34.622 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 5720, 2024-04-21T10:41:34.622 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 5720, 2024-04-21T10:41:34.622 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 774, 2024-04-21T10:41:34.622 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2024-04-21T10:41:34.625 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 282, 2024-04-21T10:41:34.625 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 15, 2024-04-21T10:41:34.625 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 14, 2024-04-21T10:41:34.625 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 14, 2024-04-21T10:41:34.626 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 1, 2024-04-21T10:41:34.626 INFO:teuthology.orchestra.run.smithi190.stdout: "segmjr": 1, 2024-04-21T10:41:34.626 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2024-04-21T10:41:34.626 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 1, 2024-04-21T10:41:34.626 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 13576516, 2024-04-21T10:41:34.626 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 14596701, 2024-04-21T10:41:34.626 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2024-04-21T10:41:34.626 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2024-04-21T10:41:34.627 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 241, 2024-04-21T10:41:34.627 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.482645681, 2024-04-21T10:41:34.627 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.006152056 2024-04-21T10:41:34.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.627 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2024-04-21T10:41:34.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.627 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2024-04-21T10:41:34.627 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 1257, 2024-04-21T10:41:34.628 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 2263, 2024-04-21T10:41:34.628 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 1006, 2024-04-21T10:41:34.628 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 119, 2024-04-21T10:41:34.628 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 135, 2024-04-21T10:41:34.628 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 16, 2024-04-21T10:41:34.628 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 1560, 2024-04-21T10:41:34.628 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 2877, 2024-04-21T10:41:34.628 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 1317, 2024-04-21T10:41:34.629 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 15, 2024-04-21T10:41:34.629 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 2236, 2024-04-21T10:41:34.629 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 2221, 2024-04-21T10:41:34.629 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 82132, 2024-04-21T10:41:34.629 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 231800 2024-04-21T10:41:34.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.629 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2024-04-21T10:41:34.629 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 6357, 2024-04-21T10:41:34.629 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 27, 2024-04-21T10:41:34.630 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 4655, 2024-04-21T10:41:34.630 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 40, 2024-04-21T10:41:34.630 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 1651, 2024-04-21T10:41:34.630 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2024-04-21T10:41:34.630 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 680, 2024-04-21T10:41:34.630 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.113465518, 2024-04-21T10:41:34.630 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001637449 2024-04-21T10:41:34.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.631 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2024-04-21T10:41:34.631 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 118, 2024-04-21T10:41:34.631 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.028721435, 2024-04-21T10:41:34.631 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000243401 2024-04-21T10:41:34.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.631 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2024-04-21T10:41:34.631 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.631 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.632 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.632 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2024-04-21T10:41:34.632 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3, 2024-04-21T10:41:34.632 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.003989084, 2024-04-21T10:41:34.632 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001329694 2024-04-21T10:41:34.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.632 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2024-04-21T10:41:34.633 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2482, 2024-04-21T10:41:34.633 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 4.431708713, 2024-04-21T10:41:34.633 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001785539 2024-04-21T10:41:34.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.633 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2024-04-21T10:41:34.633 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.633 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.633 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.634 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2024-04-21T10:41:34.634 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2024-04-21T10:41:34.634 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000480746, 2024-04-21T10:41:34.634 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000480746 2024-04-21T10:41:34.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.634 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2024-04-21T10:41:34.634 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.634 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.635 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.635 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2024-04-21T10:41:34.635 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.635 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.635 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.635 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2024-04-21T10:41:34.636 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.636 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.636 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.636 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2024-04-21T10:41:34.637 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 11, 2024-04-21T10:41:34.637 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001699653, 2024-04-21T10:41:34.637 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000154513 2024-04-21T10:41:34.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.637 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2024-04-21T10:41:34.637 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 26, 2024-04-21T10:41:34.637 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 8.827579644, 2024-04-21T10:41:34.638 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.339522294 2024-04-21T10:41:34.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.638 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2024-04-21T10:41:34.638 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.638 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.638 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.639 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2024-04-21T10:41:34.639 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 11, 2024-04-21T10:41:34.639 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.055004537, 2024-04-21T10:41:34.639 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.005000412 2024-04-21T10:41:34.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.639 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2024-04-21T10:41:34.639 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 272, 2024-04-21T10:41:34.639 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.051567398, 2024-04-21T10:41:34.639 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000189586 2024-04-21T10:41:34.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.640 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2024-04-21T10:41:34.640 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 6, 2024-04-21T10:41:34.640 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.010565260, 2024-04-21T10:41:34.640 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001760876 2024-04-21T10:41:34.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.640 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2024-04-21T10:41:34.640 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 20, 2024-04-21T10:41:34.641 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.073337008, 2024-04-21T10:41:34.641 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.003666850 2024-04-21T10:41:34.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.641 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2024-04-21T10:41:34.641 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.641 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.641 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.641 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2024-04-21T10:41:34.642 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.642 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.642 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.642 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2024-04-21T10:41:34.642 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.642 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.642 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.643 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2024-04-21T10:41:34.643 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5, 2024-04-21T10:41:34.643 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.021610159, 2024-04-21T10:41:34.643 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.004322031 2024-04-21T10:41:34.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.643 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2024-04-21T10:41:34.643 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.644 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.644 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.644 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2024-04-21T10:41:34.644 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2024-04-21T10:41:34.644 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.001482951, 2024-04-21T10:41:34.644 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001482951 2024-04-21T10:41:34.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.645 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2024-04-21T10:41:34.645 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.645 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.645 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.645 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2024-04-21T10:41:34.645 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.645 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.645 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.646 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2024-04-21T10:41:34.646 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.646 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.646 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.646 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2024-04-21T10:41:34.647 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 8, 2024-04-21T10:41:34.647 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.018924639, 2024-04-21T10:41:34.647 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002365579 2024-04-21T10:41:34.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.647 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2024-04-21T10:41:34.647 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2024-04-21T10:41:34.647 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000689943, 2024-04-21T10:41:34.647 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000689943 2024-04-21T10:41:34.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.648 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2024-04-21T10:41:34.648 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 953, 2024-04-21T10:41:34.648 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 2.170130467, 2024-04-21T10:41:34.648 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002277156 2024-04-21T10:41:34.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.648 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2024-04-21T10:41:34.648 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2024-04-21T10:41:34.648 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2024-04-21T10:41:34.648 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.649 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.649 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.649 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:34.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.649 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2024-04-21T10:41:34.649 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 2, 2024-04-21T10:41:34.649 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 6, 2024-04-21T10:41:34.649 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 4, 2024-04-21T10:41:34.650 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 2, 2024-04-21T10:41:34.650 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2024-04-21T10:41:34.650 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 38, 2024-04-21T10:41:34.650 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 19, 2024-04-21T10:41:34.650 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 634, 2024-04-21T10:41:34.650 INFO:teuthology.orchestra.run.smithi190.stdout: "mdthresh_evicted": 0 2024-04-21T10:41:34.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.650 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-04-21T10:41:34.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 360, 2024-04-21T10:41:34.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 360, 2024-04-21T10:41:34.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T10:41:34.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-04-21T10:41:34.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T10:41:34.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-04-21T10:41:34.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T10:41:34.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T10:41:34.651 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T10:41:34.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T10:41:34.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T10:41:34.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-04-21T10:41:34.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T10:41:34.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T10:41:34.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-04-21T10:41:34.652 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-04-21T10:41:34.653 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-04-21T10:41:34.653 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-04-21T10:41:34.653 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T10:41:34.653 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T10:41:34.653 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T10:41:34.653 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T10:41:34.653 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T10:41:34.654 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-04-21T10:41:34.654 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-04-21T10:41:34.654 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-04-21T10:41:34.654 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T10:41:34.654 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T10:41:34.654 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-04-21T10:41:34.654 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-04-21T10:41:34.655 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-04-21T10:41:34.655 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-04-21T10:41:34.655 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T10:41:34.655 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-04-21T10:41:34.655 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T10:41:34.655 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-04-21T10:41:34.655 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 348385, 2024-04-21T10:41:34.655 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 1336, 2024-04-21T10:41:34.656 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-04-21T10:41:34.656 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-04-21T10:41:34.656 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-04-21T10:41:34.656 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-04-21T10:41:34.656 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-04-21T10:41:34.656 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-04-21T10:41:34.656 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-04-21T10:41:34.656 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-04-21T10:41:34.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 8136, 2024-04-21T10:41:34.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 170, 2024-04-21T10:41:34.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T10:41:34.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-04-21T10:41:34.657 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-04-21T10:41:34.657 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-04-21T10:41:34.657 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 4306423, 2024-04-21T10:41:34.657 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 53770, 2024-04-21T10:41:34.657 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-04-21T10:41:34.658 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-04-21T10:41:34.658 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-04-21T10:41:34.658 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-04-21T10:41:34.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.658 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2024-04-21T10:41:34.658 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2024-04-21T10:41:34.658 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2024-04-21T10:41:34.658 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 4563, 2024-04-21T10:41:34.659 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 12925905, 2024-04-21T10:41:34.659 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 37, 2024-04-21T10:41:34.659 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 4563, 2024-04-21T10:41:34.659 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2024-04-21T10:41:34.659 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4563, 2024-04-21T10:41:34.659 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 243.911071995, 2024-04-21T10:41:34.659 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.053454103 2024-04-21T10:41:34.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.660 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2024-04-21T10:41:34.660 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2024-04-21T10:41:34.660 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4563, 2024-04-21T10:41:34.660 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 9961 2024-04-21T10:41:34.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.660 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 4563, 2024-04-21T10:41:34.660 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 17, 2024-04-21T10:41:34.660 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 4546, 2024-04-21T10:41:34.660 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2024-04-21T10:41:34.661 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2024-04-21T10:41:34.661 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 237, 2024-04-21T10:41:34.661 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 2431, 2024-04-21T10:41:34.661 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 17, 2024-04-21T10:41:34.661 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 258, 2024-04-21T10:41:34.661 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 63, 2024-04-21T10:41:34.661 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2024-04-21T10:41:34.661 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2024-04-21T10:41:34.661 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2024-04-21T10:41:34.662 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2024-04-21T10:41:34.662 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 1475, 2024-04-21T10:41:34.662 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2024-04-21T10:41:34.662 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2024-04-21T10:41:34.662 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2024-04-21T10:41:34.662 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 0, 2024-04-21T10:41:34.662 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 4863, 2024-04-21T10:41:34.662 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2024-04-21T10:41:34.663 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2024-04-21T10:41:34.663 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2024-04-21T10:41:34.663 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2024-04-21T10:41:34.663 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2024-04-21T10:41:34.663 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2024-04-21T10:41:34.663 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T10:41:34.663 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2024-04-21T10:41:34.663 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2024-04-21T10:41:34.663 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2024-04-21T10:41:34.664 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2024-04-21T10:41:34.664 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2024-04-21T10:41:34.664 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2024-04-21T10:41:34.664 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2024-04-21T10:41:34.664 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2024-04-21T10:41:34.664 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2024-04-21T10:41:34.664 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2024-04-21T10:41:34.664 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2024-04-21T10:41:34.664 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2024-04-21T10:41:34.665 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2024-04-21T10:41:34.665 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2024-04-21T10:41:34.665 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2024-04-21T10:41:34.665 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2024-04-21T10:41:34.665 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2024-04-21T10:41:34.665 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2024-04-21T10:41:34.665 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2024-04-21T10:41:34.665 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 100, 2024-04-21T10:41:34.665 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2024-04-21T10:41:34.666 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 14, 2024-04-21T10:41:34.666 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2024-04-21T10:41:34.666 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2024-04-21T10:41:34.666 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2024-04-21T10:41:34.666 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2024-04-21T10:41:34.666 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 529, 2024-04-21T10:41:34.666 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 0, 2024-04-21T10:41:34.666 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 88 2024-04-21T10:41:34.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.667 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2024-04-21T10:41:34.667 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2024-04-21T10:41:34.667 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2024-04-21T10:41:34.667 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2024-04-21T10:41:34.667 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2024-04-21T10:41:34.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.667 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2024-04-21T10:41:34.667 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 46 2024-04-21T10:41:34.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.668 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2024-04-21T10:41:34.668 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2024-04-21T10:41:34.668 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 10, 2024-04-21T10:41:34.668 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2024-04-21T10:41:34.668 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 10, 2024-04-21T10:41:34.668 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed_ops": 959, 2024-04-21T10:41:34.668 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 959, 2024-04-21T10:41:34.668 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2024-04-21T10:41:34.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.669 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T10:41:34.669 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:34.669 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-04-21T10:41:34.669 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:34.669 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 17856, 2024-04-21T10:41:34.669 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 7350203, 2024-04-21T10:41:34.669 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:34.669 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 17856, 2024-04-21T10:41:34.670 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-04-21T10:41:34.670 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-04-21T10:41:34.670 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 17856, 2024-04-21T10:41:34.670 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 7350203, 2024-04-21T10:41:34.670 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:34.670 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.670 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.670 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.670 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:34.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.671 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2024-04-21T10:41:34.671 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:34.671 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-04-21T10:41:34.671 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:34.671 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-04-21T10:41:34.671 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-04-21T10:41:34.671 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:34.672 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:34.672 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 4563, 2024-04-21T10:41:34.672 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 13022764, 2024-04-21T10:41:34.672 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 4563, 2024-04-21T10:41:34.672 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 13022764, 2024-04-21T10:41:34.672 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:34.673 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.673 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.673 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.673 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:34.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.673 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2024-04-21T10:41:34.673 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:34.673 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2024-04-21T10:41:34.673 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:34.674 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2024-04-21T10:41:34.674 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2024-04-21T10:41:34.674 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:34.674 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2024-04-21T10:41:34.674 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 4563, 2024-04-21T10:41:34.674 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 4563, 2024-04-21T10:41:34.674 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 4563, 2024-04-21T10:41:34.674 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 4563, 2024-04-21T10:41:34.674 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:34.675 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.675 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.675 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.675 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:34.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.675 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2024-04-21T10:41:34.675 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:34.675 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-04-21T10:41:34.675 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:34.676 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 959, 2024-04-21T10:41:34.676 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 96859, 2024-04-21T10:41:34.676 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:34.676 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 959, 2024-04-21T10:41:34.676 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-04-21T10:41:34.676 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-04-21T10:41:34.676 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 17, 2024-04-21T10:41:34.676 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 96859, 2024-04-21T10:41:34.676 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:34.677 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.677 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.677 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.677 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:34.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:34.677 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x559b06316500": { 2024-04-21T10:41:34.677 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:34.677 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2024-04-21T10:41:34.678 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:34.678 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 6494, 2024-04-21T10:41:34.678 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 10402397, 2024-04-21T10:41:34.678 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:34.678 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 6494, 2024-04-21T10:41:34.678 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-04-21T10:41:34.678 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-04-21T10:41:34.678 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 241, 2024-04-21T10:41:34.678 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 10402397, 2024-04-21T10:41:34.679 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:34.679 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:34.679 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:34.679 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:34.679 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:34.679 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:34.679 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-04-21T10:41:35.378 INFO:tasks.check_counter:Daemon mds.i mds.exported=3 2024-04-21T10:41:35.378 INFO:tasks.check_counter:Daemon mds.i mds.imported=67 2024-04-21T10:41:35.379 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=208 2024-04-21T10:41:35.379 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=48 2024-04-21T10:41:35.379 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-21T10:41:35.379 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=11 2024-04-21T10:41:35.379 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=5 2024-04-21T10:41:35.379 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-21T10:41:35.379 DEBUG:tasks.check_counter:Getting stats from l 2024-04-21T10:41:35.380 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-21T10:41:35.623 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T10:41:35.863 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:35 smithi190 ceph-mon[31423]: pgmap v331: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:41:35.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:35 smithi026 ceph-mon[27965]: pgmap v331: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:41:35.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:35 smithi169 ceph-mon[31469]: pgmap v331: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-21T10:41:37.337 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-04-21T10:41:37.338 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T10:41:37.338 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 4, 2024-04-21T10:41:37.338 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 4, 2024-04-21T10:41:37.338 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 3040, 2024-04-21T10:41:37.338 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 448, 2024-04-21T10:41:37.339 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 4, 2024-04-21T10:41:37.339 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 0, 2024-04-21T10:41:37.339 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.001360210, 2024-04-21T10:41:37.339 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.000037139, 2024-04-21T10:41:37.339 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.000477470, 2024-04-21T10:41:37.339 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T10:41:37.339 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:37.339 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3, 2024-04-21T10:41:37.340 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000681422, 2024-04-21T10:41:37.340 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000227140 2024-04-21T10:41:37.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:37.340 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:37.340 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:37.340 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:37.340 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:37.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:37.341 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-21T10:41:37.341 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T10:41:37.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:37.341 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T10:41:37.341 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 2, 2024-04-21T10:41:37.341 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 162, 2024-04-21T10:41:37.341 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 320, 2024-04-21T10:41:37.341 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 184720, 2024-04-21T10:41:37.342 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2024-04-21T10:41:37.342 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2024-04-21T10:41:37.342 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.010771468, 2024-04-21T10:41:37.342 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.007252583, 2024-04-21T10:41:37.342 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.001157949, 2024-04-21T10:41:37.342 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T10:41:37.342 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:37.342 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 159, 2024-04-21T10:41:37.343 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.015936870, 2024-04-21T10:41:37.343 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000100231 2024-04-21T10:41:37.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:37.343 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:37.343 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:37.343 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:37.344 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:37.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:37.344 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T10:41:37.344 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 184720 2024-04-21T10:41:37.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:37.344 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T10:41:37.344 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 222, 2024-04-21T10:41:37.344 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 212, 2024-04-21T10:41:37.344 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 1451184, 2024-04-21T10:41:37.345 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 130464, 2024-04-21T10:41:37.345 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 2, 2024-04-21T10:41:37.345 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 1, 2024-04-21T10:41:37.345 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.047443458, 2024-04-21T10:41:37.345 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.014458009, 2024-04-21T10:41:37.345 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.035584744, 2024-04-21T10:41:37.345 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.005415721, 2024-04-21T10:41:37.345 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T10:41:37.346 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 211, 2024-04-21T10:41:37.346 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.011915233, 2024-04-21T10:41:37.346 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000056470 2024-04-21T10:41:37.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:37.346 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2024-04-21T10:41:37.346 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:37.346 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:37.346 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:37.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:37.347 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 1451184, 2024-04-21T10:41:37.347 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 130464 2024-04-21T10:41:37.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:37.347 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2024-04-21T10:41:37.347 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 0, 2024-04-21T10:41:37.347 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2024-04-21T10:41:37.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:37.348 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2024-04-21T10:41:37.348 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2024-04-21T10:41:37.348 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2024-04-21T10:41:37.348 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T10:41:37.348 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2024-04-21T10:41:37.348 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T10:41:37.348 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2024-04-21T10:41:37.348 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T10:41:37.348 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T10:41:37.349 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T10:41:37.349 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T10:41:37.349 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T10:41:37.349 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2024-04-21T10:41:37.349 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T10:41:37.349 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T10:41:37.349 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2024-04-21T10:41:37.349 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2024-04-21T10:41:37.349 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2024-04-21T10:41:37.350 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2024-04-21T10:41:37.350 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T10:41:37.350 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T10:41:37.350 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T10:41:37.350 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T10:41:37.350 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T10:41:37.350 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2024-04-21T10:41:37.350 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2024-04-21T10:41:37.351 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2024-04-21T10:41:37.351 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T10:41:37.351 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T10:41:37.351 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2024-04-21T10:41:37.351 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2024-04-21T10:41:37.351 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2024-04-21T10:41:37.351 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2024-04-21T10:41:37.351 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T10:41:37.352 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2024-04-21T10:41:37.352 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T10:41:37.352 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2024-04-21T10:41:37.352 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 148967, 2024-04-21T10:41:37.352 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 29, 2024-04-21T10:41:37.352 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2024-04-21T10:41:37.352 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2024-04-21T10:41:37.352 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2024-04-21T10:41:37.352 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2024-04-21T10:41:37.353 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2024-04-21T10:41:37.353 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2024-04-21T10:41:37.353 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2024-04-21T10:41:37.353 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2024-04-21T10:41:37.353 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 0, 2024-04-21T10:41:37.353 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 0, 2024-04-21T10:41:37.353 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T10:41:37.353 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2024-04-21T10:41:37.354 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2024-04-21T10:41:37.354 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2024-04-21T10:41:37.354 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 928, 2024-04-21T10:41:37.354 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 2, 2024-04-21T10:41:37.354 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2024-04-21T10:41:37.354 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2024-04-21T10:41:37.354 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2024-04-21T10:41:37.354 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2024-04-21T10:41:37.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-04-21T10:41:37.355 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T10:41:37.355 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2024-04-21T10:41:37.355 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2024-04-21T10:41:37.355 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2024-04-21T10:41:37.355 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 224, 2024-04-21T10:41:37.355 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 1419860, 2024-04-21T10:41:37.355 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2024-04-21T10:41:37.355 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 224, 2024-04-21T10:41:37.356 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2024-04-21T10:41:37.356 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2024-04-21T10:41:37.356 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 224, 2024-04-21T10:41:37.356 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1419860, 2024-04-21T10:41:37.356 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2024-04-21T10:41:37.356 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2024-04-21T10:41:37.356 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2024-04-21T10:41:37.356 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2024-04-21T10:41:37.356 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:37.356 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-04-21T10:41:37.357 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-04-21T10:41:37.658 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:37 smithi190 ceph-mon[31423]: pgmap v332: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:41:37.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:37 smithi026 ceph-mon[27965]: pgmap v332: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:41:37.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:37 smithi169 ceph-mon[31469]: pgmap v332: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-21T10:41:38.011 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-21T10:41:38.012 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-21T10:41:38.012 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-21T10:41:38.012 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-21T10:41:38.012 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-21T10:41:38.012 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-21T10:41:38.012 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-21T10:41:38.012 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-21T10:41:38.013 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-21T10:41:38.030 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-21T10:41:38.030 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:41:38.031 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T10:41:38.052 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-21T10:41:38.052 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T10:41:38.111 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:41:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.269+0000 7f5247e2c640 1 -- 172.21.15.26:0/433506630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5240102b50 msgr2=0x7f5240106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.269+0000 7f5247e2c640 1 --2- 172.21.15.26:0/433506630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5240102b50 0x7f5240106b90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5234009900 tx=0x7f523402bc30 comp rx=0 tx=0).stop 2024-04-21T10:41:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.269+0000 7f5247e2c640 1 -- 172.21.15.26:0/433506630 shutdown_connections 2024-04-21T10:41:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.269+0000 7f5247e2c640 1 --2- 172.21.15.26:0/433506630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5240107270 0x7f524010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.269+0000 7f5247e2c640 1 --2- 172.21.15.26:0/433506630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5240102b50 0x7f5240106b90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.269+0000 7f5247e2c640 1 --2- 172.21.15.26:0/433506630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52401021a0 0x7f5240102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.269+0000 7f5247e2c640 1 -- 172.21.15.26:0/433506630 >> 172.21.15.26:0/433506630 conn(0x7f52400793a0 msgr2=0x7f52400797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.269+0000 7f5247e2c640 1 -- 172.21.15.26:0/433506630 shutdown_connections 2024-04-21T10:41:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.269+0000 7f5247e2c640 1 -- 172.21.15.26:0/433506630 wait complete. 2024-04-21T10:41:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f5247e2c640 1 Processor -- start 2024-04-21T10:41:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f5247e2c640 1 -- start start 2024-04-21T10:41:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f5247e2c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52401021a0 0x7f5240198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f5247e2c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5240107270 0x7f5240198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f5247e2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524019cff0 0x7f524019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f5247e2c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f52401121b0 con 0x7f524019cff0 2024-04-21T10:41:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f5247e2c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5240112030 con 0x7f5240107270 2024-04-21T10:41:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f5247e2c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5240112330 con 0x7f52401021a0 2024-04-21T10:41:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f52463c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524019cff0 0x7f524019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f52463c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524019cff0 0x7f524019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55390/0 (socket says 172.21.15.26:55390) 2024-04-21T10:41:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f52463c9640 1 -- 172.21.15.26:0/2173175874 learned_addr learned my addr 172.21.15.26:0/2173175874 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f5245bc8640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52401021a0 0x7f5240198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f52453c7640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5240107270 0x7f5240198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f52463c9640 1 -- 172.21.15.26:0/2173175874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52401021a0 msgr2=0x7f5240198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f52463c9640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52401021a0 0x7f5240198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f52463c9640 1 -- 172.21.15.26:0/2173175874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5240107270 msgr2=0x7f5240198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f52463c9640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5240107270 0x7f5240198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f52463c9640 1 -- 172.21.15.26:0/2173175874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524019dae0 con 0x7f524019cff0 2024-04-21T10:41:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.270+0000 7f52453c7640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5240107270 0x7f5240198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:41:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.271+0000 7f52463c9640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524019cff0 0x7f524019d4a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5238009fd0 tx=0x7f523800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.271+0000 7f5232ffd640 1 -- 172.21.15.26:0/2173175874 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238017070 con 0x7f524019cff0 2024-04-21T10:41:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.271+0000 7f5232ffd640 1 -- 172.21.15.26:0/2173175874 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5238003e20 con 0x7f524019cff0 2024-04-21T10:41:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.271+0000 7f5247e2c640 1 -- 172.21.15.26:0/2173175874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524019ddd0 con 0x7f524019cff0 2024-04-21T10:41:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.271+0000 7f5232ffd640 1 -- 172.21.15.26:0/2173175874 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238010460 con 0x7f524019cff0 2024-04-21T10:41:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.271+0000 7f5247e2c640 1 -- 172.21.15.26:0/2173175874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52401a55c0 con 0x7f524019cff0 2024-04-21T10:41:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.272+0000 7f5247e2c640 1 -- 172.21.15.26:0/2173175874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5240069230 con 0x7f524019cff0 2024-04-21T10:41:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.273+0000 7f5232ffd640 1 -- 172.21.15.26:0/2173175874 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5238003970 con 0x7f524019cff0 2024-04-21T10:41:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.274+0000 7f5232ffd640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f522c0a3da0 0x7f522c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.274+0000 7f5245bc8640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f522c0a3da0 0x7f522c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:38.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.274+0000 7f5232ffd640 1 -- 172.21.15.26:0/2173175874 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7f52380d0450 con 0x7f524019cff0 2024-04-21T10:41:38.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.275+0000 7f5245bc8640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f522c0a3da0 0x7f522c0a6260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f523c009bd0 tx=0x7f523c005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:38.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.278+0000 7f5232ffd640 1 -- 172.21.15.26:0/2173175874 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f523809cb50 con 0x7f524019cff0 2024-04-21T10:41:38.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.470+0000 7f5247e2c640 1 -- 172.21.15.26:0/2173175874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5240102b50 con 0x7f524019cff0 2024-04-21T10:41:38.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.472+0000 7f5232ffd640 1 -- 172.21.15.26:0/2173175874 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23170 (secure 0 0 0) 0x7f523809c450 con 0x7f524019cff0 2024-04-21T10:41:38.472 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:41:38.472 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":100,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:41:13.960179+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3802401516":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/3125048023":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4245538524":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2797499963":"2024-04-22T10:30:57.800969+0000","172.21.15.26:6801/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:41:38.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.474+0000 7f5247e2c640 1 -- 172.21.15.26:0/2173175874 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f522c0a3da0 msgr2=0x7f522c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.474+0000 7f5247e2c640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f522c0a3da0 0x7f522c0a6260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f523c009bd0 tx=0x7f523c005f90 comp rx=0 tx=0).stop 2024-04-21T10:41:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.474+0000 7f5247e2c640 1 -- 172.21.15.26:0/2173175874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524019cff0 msgr2=0x7f524019d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.474+0000 7f5247e2c640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524019cff0 0x7f524019d4a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5238009fd0 tx=0x7f523800b260 comp rx=0 tx=0).stop 2024-04-21T10:41:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.474+0000 7f5247e2c640 1 -- 172.21.15.26:0/2173175874 shutdown_connections 2024-04-21T10:41:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.474+0000 7f5247e2c640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f522c0a3da0 0x7f522c0a6260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.474+0000 7f5247e2c640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524019cff0 0x7f524019d4a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.474+0000 7f5247e2c640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5240107270 0x7f5240198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.474+0000 7f5247e2c640 1 --2- 172.21.15.26:0/2173175874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52401021a0 0x7f5240198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.474+0000 7f5247e2c640 1 -- 172.21.15.26:0/2173175874 >> 172.21.15.26:0/2173175874 conn(0x7f52400793a0 msgr2=0x7f52400fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.474+0000 7f5247e2c640 1 -- 172.21.15.26:0/2173175874 shutdown_connections 2024-04-21T10:41:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.474+0000 7f5247e2c640 1 -- 172.21.15.26:0/2173175874 wait complete. 2024-04-21T10:41:38.485 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T10:24:28.176891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T10:28:59.417687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T10:29:00.234808+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T10:41:38.485 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T10:41:38.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.591+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1090952548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe048101a50 msgr2=0x7fe048105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.591+0000 7fe04f6f2640 1 --2- 172.21.15.26:0/1090952548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe048101a50 0x7fe048105a90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe038009900 tx=0x7fe03802bc30 comp rx=0 tx=0).stop 2024-04-21T10:41:38.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.591+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1090952548 shutdown_connections 2024-04-21T10:41:38.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.591+0000 7fe04f6f2640 1 --2- 172.21.15.26:0/1090952548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe048106170 0x7fe04810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.591+0000 7fe04f6f2640 1 --2- 172.21.15.26:0/1090952548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe048101a50 0x7fe048105a90 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.591+0000 7fe04f6f2640 1 --2- 172.21.15.26:0/1090952548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0481010a0 0x7fe048101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.591+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1090952548 >> 172.21.15.26:0/1090952548 conn(0x7fe048077eb0 msgr2=0x7fe0480782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.591+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1090952548 shutdown_connections 2024-04-21T10:41:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.591+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1090952548 wait complete. 2024-04-21T10:41:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04f6f2640 1 Processor -- start 2024-04-21T10:41:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04f6f2640 1 -- start start 2024-04-21T10:41:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04f6f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0481010a0 0x7fe0481975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:38.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04f6f2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe048106170 0x7fe048197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:38.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04f6f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04819be90 0x7fe04819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:38.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04f6f2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe048111090 con 0x7fe04819be90 2024-04-21T10:41:38.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04f6f2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe048110f10 con 0x7fe0481010a0 2024-04-21T10:41:38.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04f6f2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe048111210 con 0x7fe048106170 2024-04-21T10:41:38.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04dc8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04819be90 0x7fe04819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:38.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04dc8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04819be90 0x7fe04819c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55406/0 (socket says 172.21.15.26:55406) 2024-04-21T10:41:38.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04dc8f640 1 -- 172.21.15.26:0/1617067736 learned_addr learned my addr 172.21.15.26:0/1617067736 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:38.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04d48e640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0481010a0 0x7fe0481975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04cc8d640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe048106170 0x7fe048197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04dc8f640 1 -- 172.21.15.26:0/1617067736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe048106170 msgr2=0x7fe048197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04dc8f640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe048106170 0x7fe048197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04dc8f640 1 -- 172.21.15.26:0/1617067736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0481010a0 msgr2=0x7fe0481975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04dc8f640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0481010a0 0x7fe0481975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.592+0000 7fe04dc8f640 1 -- 172.21.15.26:0/1617067736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04819c980 con 0x7fe04819be90 2024-04-21T10:41:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.593+0000 7fe04dc8f640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04819be90 0x7fe04819c340 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe04000cce0 tx=0x7fe04000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:38.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.593+0000 7fe03e7fc640 1 -- 172.21.15.26:0/1617067736 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04000f450 con 0x7fe04819be90 2024-04-21T10:41:38.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.593+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1617067736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04819cc70 con 0x7fe04819be90 2024-04-21T10:41:38.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.593+0000 7fe03e7fc640 1 -- 172.21.15.26:0/1617067736 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe04000faa0 con 0x7fe04819be90 2024-04-21T10:41:38.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.593+0000 7fe03e7fc640 1 -- 172.21.15.26:0/1617067736 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040018b70 con 0x7fe04819be90 2024-04-21T10:41:38.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.593+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1617067736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0481a4430 con 0x7fe04819be90 2024-04-21T10:41:38.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.594+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1617067736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0480691c0 con 0x7fe04819be90 2024-04-21T10:41:38.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.595+0000 7fe03e7fc640 1 -- 172.21.15.26:0/1617067736 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fe040018d10 con 0x7fe04819be90 2024-04-21T10:41:38.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.596+0000 7fe03e7fc640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fe0340a3d20 0x7fe0340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:38.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.596+0000 7fe03e7fc640 1 -- 172.21.15.26:0/1617067736 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7fe0400d0c10 con 0x7fe04819be90 2024-04-21T10:41:38.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.596+0000 7fe04d48e640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fe0340a3d20 0x7fe0340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:38.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.596+0000 7fe04d48e640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fe0340a3d20 0x7fe0340a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe04400bbe0 tx=0x7fe04400b390 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:38.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.599+0000 7fe03e7fc640 1 -- 172.21.15.26:0/1617067736 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe040014030 con 0x7fe04819be90 2024-04-21T10:41:38.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.786+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1617067736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe048101a50 con 0x7fe04819be90 2024-04-21T10:41:38.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.786+0000 7fe03e7fc640 1 -- 172.21.15.26:0/1617067736 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fe04009d320 con 0x7fe04819be90 2024-04-21T10:41:38.786 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-21T10:41:38.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.788+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1617067736 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fe0340a3d20 msgr2=0x7fe0340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.788+0000 7fe04f6f2640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fe0340a3d20 0x7fe0340a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe04400bbe0 tx=0x7fe04400b390 comp rx=0 tx=0).stop 2024-04-21T10:41:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.788+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1617067736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04819be90 msgr2=0x7fe04819c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.788+0000 7fe04f6f2640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04819be90 0x7fe04819c340 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe04000cce0 tx=0x7fe04000a620 comp rx=0 tx=0).stop 2024-04-21T10:41:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.788+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1617067736 shutdown_connections 2024-04-21T10:41:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.788+0000 7fe04f6f2640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fe0340a3d20 0x7fe0340a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.788+0000 7fe04f6f2640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04819be90 0x7fe04819c340 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.788+0000 7fe04f6f2640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe048106170 0x7fe048197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.788+0000 7fe04f6f2640 1 --2- 172.21.15.26:0/1617067736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0481010a0 0x7fe0481975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.788+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1617067736 >> 172.21.15.26:0/1617067736 conn(0x7fe048077eb0 msgr2=0x7fe0481005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:38.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.788+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1617067736 shutdown_connections 2024-04-21T10:41:38.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.788+0000 7fe04f6f2640 1 -- 172.21.15.26:0/1617067736 wait complete. 2024-04-21T10:41:38.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:38 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2173175874' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:41:38.798 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T10:41:38.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.905+0000 7f63beaf2640 1 -- 172.21.15.26:0/2472033400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8107270 msgr2=0x7f63b810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.905+0000 7f63beaf2640 1 --2- 172.21.15.26:0/2472033400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8107270 0x7f63b810ee40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009ca0 tx=0x7f63b002c0f0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.905+0000 7f63beaf2640 1 -- 172.21.15.26:0/2472033400 shutdown_connections 2024-04-21T10:41:38.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.905+0000 7f63beaf2640 1 --2- 172.21.15.26:0/2472033400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8107270 0x7f63b810ee40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.905+0000 7f63beaf2640 1 --2- 172.21.15.26:0/2472033400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b806b1f0 0x7f63b8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.905+0000 7f63beaf2640 1 --2- 172.21.15.26:0/2472033400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b806a840 0x7f63b806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.905+0000 7f63beaf2640 1 -- 172.21.15.26:0/2472033400 >> 172.21.15.26:0/2472033400 conn(0x7f63b8077eb0 msgr2=0x7f63b80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:38.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.905+0000 7f63beaf2640 1 -- 172.21.15.26:0/2472033400 shutdown_connections 2024-04-21T10:41:38.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.905+0000 7f63beaf2640 1 -- 172.21.15.26:0/2472033400 wait complete. 2024-04-21T10:41:38.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.905+0000 7f63beaf2640 1 Processor -- start 2024-04-21T10:41:38.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.905+0000 7f63beaf2640 1 -- start start 2024-04-21T10:41:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63beaf2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b806a840 0x7f63b81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63beaf2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8107270 0x7f63b8198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63beaf2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b819cf70 0x7f63b819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63beaf2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81121d0 con 0x7f63b8107270 2024-04-21T10:41:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63beaf2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8112050 con 0x7f63b806a840 2024-04-21T10:41:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63beaf2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8112350 con 0x7f63b819cf70 2024-04-21T10:41:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63b7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8107270 0x7f63b8198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63b7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8107270 0x7f63b8198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55420/0 (socket says 172.21.15.26:55420) 2024-04-21T10:41:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63b7fff640 1 -- 172.21.15.26:0/3043129266 learned_addr learned my addr 172.21.15.26:0/3043129266 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63bc88e640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b806a840 0x7f63b81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63bd08f640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b819cf70 0x7f63b819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63b7fff640 1 -- 172.21.15.26:0/3043129266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b819cf70 msgr2=0x7f63b819d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63b7fff640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b819cf70 0x7f63b819d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63b7fff640 1 -- 172.21.15.26:0/3043129266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b806a840 msgr2=0x7f63b81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63b7fff640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b806a840 0x7f63b81986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:38.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63b7fff640 1 -- 172.21.15.26:0/3043129266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b819da60 con 0x7f63b8107270 2024-04-21T10:41:38.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63bc88e640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b806a840 0x7f63b81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:41:38.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63b7fff640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8107270 0x7f63b8198bf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00d770 tx=0x7f63ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:38.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63b5ffb640 1 -- 172.21.15.26:0/3043129266 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac017070 con 0x7f63b8107270 2024-04-21T10:41:38.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63b5ffb640 1 -- 172.21.15.26:0/3043129266 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ac009c30 con 0x7f63b8107270 2024-04-21T10:41:38.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63beaf2640 1 -- 172.21.15.26:0/3043129266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b819dd50 con 0x7f63b8107270 2024-04-21T10:41:38.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.906+0000 7f63b5ffb640 1 -- 172.21.15.26:0/3043129266 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac00f620 con 0x7f63b8107270 2024-04-21T10:41:38.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.907+0000 7f63beaf2640 1 -- 172.21.15.26:0/3043129266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b81a5590 con 0x7f63b8107270 2024-04-21T10:41:38.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.907+0000 7f63beaf2640 1 -- 172.21.15.26:0/3043129266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b80691c0 con 0x7f63b8107270 2024-04-21T10:41:38.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.909+0000 7f63b5ffb640 1 -- 172.21.15.26:0/3043129266 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f63ac01a070 con 0x7f63b8107270 2024-04-21T10:41:38.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.909+0000 7f63b5ffb640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f63a00a3da0 0x7f63a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:38.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.909+0000 7f63bc88e640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f63a00a3da0 0x7f63a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:38.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.909+0000 7f63b5ffb640 1 -- 172.21.15.26:0/3043129266 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7f63ac0d02d0 con 0x7f63b8107270 2024-04-21T10:41:38.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.910+0000 7f63bc88e640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f63a00a3da0 0x7f63a00a6260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009b50 tx=0x7f63a8005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:38.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:38.913+0000 7f63b5ffb640 1 -- 172.21.15.26:0/3043129266 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63ac009de0 con 0x7f63b8107270 2024-04-21T10:41:38.950 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:38 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2173175874' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:41:39.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:38 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2173175874' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:41:39.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.103+0000 7f63beaf2640 1 -- 172.21.15.26:0/3043129266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f63b806b1f0 con 0x7f63b8107270 2024-04-21T10:41:39.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.104+0000 7f63b5ffb640 1 -- 172.21.15.26:0/3043129266 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f63ac09c9d0 con 0x7f63b8107270 2024-04-21T10:41:39.103 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-21T10:41:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.106+0000 7f63beaf2640 1 -- 172.21.15.26:0/3043129266 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f63a00a3da0 msgr2=0x7f63a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.106+0000 7f63beaf2640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f63a00a3da0 0x7f63a00a6260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009b50 tx=0x7f63a8005f90 comp rx=0 tx=0).stop 2024-04-21T10:41:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.106+0000 7f63beaf2640 1 -- 172.21.15.26:0/3043129266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8107270 msgr2=0x7f63b8198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.106+0000 7f63beaf2640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8107270 0x7f63b8198bf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00d770 tx=0x7f63ac00dc30 comp rx=0 tx=0).stop 2024-04-21T10:41:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.106+0000 7f63beaf2640 1 -- 172.21.15.26:0/3043129266 shutdown_connections 2024-04-21T10:41:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.106+0000 7f63beaf2640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f63a00a3da0 0x7f63a00a6260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.106+0000 7f63beaf2640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b819cf70 0x7f63b819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.106+0000 7f63beaf2640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63b8107270 0x7f63b8198bf0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.106+0000 7f63beaf2640 1 --2- 172.21.15.26:0/3043129266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b806a840 0x7f63b81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.106+0000 7f63beaf2640 1 -- 172.21.15.26:0/3043129266 >> 172.21.15.26:0/3043129266 conn(0x7f63b8077eb0 msgr2=0x7f63b80fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.106+0000 7f63beaf2640 1 -- 172.21.15.26:0/3043129266 shutdown_connections 2024-04-21T10:41:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.106+0000 7f63beaf2640 1 -- 172.21.15.26:0/3043129266 wait complete. 2024-04-21T10:41:39.116 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T10:41:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.222+0000 7fecea175640 1 -- 172.21.15.26:0/3564094661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece41010a0 msgr2=0x7fece4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.222+0000 7fecea175640 1 --2- 172.21.15.26:0/3564094661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece41010a0 0x7fece4101480 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feccc009900 tx=0x7feccc02bc30 comp rx=0 tx=0).stop 2024-04-21T10:41:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.222+0000 7fecea175640 1 -- 172.21.15.26:0/3564094661 shutdown_connections 2024-04-21T10:41:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.222+0000 7fecea175640 1 --2- 172.21.15.26:0/3564094661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece4106170 0x7fece410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.222+0000 7fecea175640 1 --2- 172.21.15.26:0/3564094661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece4101a50 0x7fece4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.222+0000 7fecea175640 1 --2- 172.21.15.26:0/3564094661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece41010a0 0x7fece4101480 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.222+0000 7fecea175640 1 -- 172.21.15.26:0/3564094661 >> 172.21.15.26:0/3564094661 conn(0x7fece4077eb0 msgr2=0x7fece40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.222+0000 7fecea175640 1 -- 172.21.15.26:0/3564094661 shutdown_connections 2024-04-21T10:41:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.222+0000 7fecea175640 1 -- 172.21.15.26:0/3564094661 wait complete. 2024-04-21T10:41:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.223+0000 7fecea175640 1 Processor -- start 2024-04-21T10:41:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.223+0000 7fecea175640 1 -- start start 2024-04-21T10:41:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.223+0000 7fecea175640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece41010a0 0x7fece4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.223+0000 7fecea175640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece4106170 0x7fece4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.223+0000 7fecea175640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece419d010 0x7fece419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.223+0000 7fecea175640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fece41121f0 con 0x7fece41010a0 2024-04-21T10:41:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.223+0000 7fecea175640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fece4112070 con 0x7fece419d010 2024-04-21T10:41:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.223+0000 7fecea175640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fece4112370 con 0x7fece4106170 2024-04-21T10:41:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.223+0000 7fece37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece41010a0 0x7fece4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.223+0000 7fece37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece41010a0 0x7fece4198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55440/0 (socket says 172.21.15.26:55440) 2024-04-21T10:41:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.223+0000 7fece37fe640 1 -- 172.21.15.26:0/2463790993 learned_addr learned my addr 172.21.15.26:0/2463790993 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fece2ffd640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece4106170 0x7fece4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fece3fff640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece419d010 0x7fece419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fece37fe640 1 -- 172.21.15.26:0/2463790993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece4106170 msgr2=0x7fece4198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fece37fe640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece4106170 0x7fece4198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fece37fe640 1 -- 172.21.15.26:0/2463790993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece419d010 msgr2=0x7fece419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fece37fe640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece419d010 0x7fece419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fece37fe640 1 -- 172.21.15.26:0/2463790993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece419db00 con 0x7fece41010a0 2024-04-21T10:41:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fece3fff640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece419d010 0x7fece419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:41:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fece37fe640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece41010a0 0x7fece4198750 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feccc005940 tx=0x7feccc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fece0ff9640 1 -- 172.21.15.26:0/2463790993 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc043020 con 0x7fece41010a0 2024-04-21T10:41:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fecea175640 1 -- 172.21.15.26:0/2463790993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece419dd90 con 0x7fece41010a0 2024-04-21T10:41:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fece0ff9640 1 -- 172.21.15.26:0/2463790993 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feccc02caf0 con 0x7fece41010a0 2024-04-21T10:41:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fece0ff9640 1 -- 172.21.15.26:0/2463790993 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc003af0 con 0x7fece41010a0 2024-04-21T10:41:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.224+0000 7fecea175640 1 -- 172.21.15.26:0/2463790993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece41a55e0 con 0x7fece41010a0 2024-04-21T10:41:39.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.225+0000 7fecea175640 1 -- 172.21.15.26:0/2463790993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece40691c0 con 0x7fece41010a0 2024-04-21T10:41:39.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.226+0000 7fece0ff9640 1 -- 172.21.15.26:0/2463790993 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7feccc046020 con 0x7fece41010a0 2024-04-21T10:41:39.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.227+0000 7fece0ff9640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fecd00a3d20 0x7fecd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:39.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.227+0000 7fece2ffd640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fecd00a3d20 0x7fecd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:39.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.227+0000 7fece0ff9640 1 -- 172.21.15.26:0/2463790993 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7feccc0f2f70 con 0x7fece41010a0 2024-04-21T10:41:39.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.227+0000 7fece2ffd640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fecd00a3d20 0x7fecd00a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fece4100a20 tx=0x7fecd4005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:39.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.230+0000 7fece0ff9640 1 -- 172.21.15.26:0/2463790993 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feccc04d020 con 0x7fece41010a0 2024-04-21T10:41:39.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.417+0000 7fecea175640 1 -- 172.21.15.26:0/2463790993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fece41093a0 con 0x7fece41010a0 2024-04-21T10:41:39.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.418+0000 7fece0ff9640 1 -- 172.21.15.26:0/2463790993 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7feccc0bf700 con 0x7fece41010a0 2024-04-21T10:41:39.417 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-21T10:41:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.420+0000 7fecea175640 1 -- 172.21.15.26:0/2463790993 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fecd00a3d20 msgr2=0x7fecd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.420+0000 7fecea175640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fecd00a3d20 0x7fecd00a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fece4100a20 tx=0x7fecd4005f90 comp rx=0 tx=0).stop 2024-04-21T10:41:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.420+0000 7fecea175640 1 -- 172.21.15.26:0/2463790993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece41010a0 msgr2=0x7fece4198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.420+0000 7fecea175640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece41010a0 0x7fece4198750 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feccc005940 tx=0x7feccc0045e0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.420+0000 7fecea175640 1 -- 172.21.15.26:0/2463790993 shutdown_connections 2024-04-21T10:41:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.420+0000 7fecea175640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fecd00a3d20 0x7fecd00a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.420+0000 7fecea175640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece419d010 0x7fece419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.420+0000 7fecea175640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fece4106170 0x7fece4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.420+0000 7fecea175640 1 --2- 172.21.15.26:0/2463790993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece41010a0 0x7fece4198750 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.420+0000 7fecea175640 1 -- 172.21.15.26:0/2463790993 >> 172.21.15.26:0/2463790993 conn(0x7fece4077eb0 msgr2=0x7fece40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.420+0000 7fecea175640 1 -- 172.21.15.26:0/2463790993 shutdown_connections 2024-04-21T10:41:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.420+0000 7fecea175640 1 -- 172.21.15.26:0/2463790993 wait complete. 2024-04-21T10:41:39.430 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T10:41:39.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.534+0000 7f3c6fe81640 1 -- 172.21.15.26:0/657038689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c68101a50 msgr2=0x7f3c68105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:39.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.534+0000 7f3c6fe81640 1 --2- 172.21.15.26:0/657038689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c68101a50 0x7f3c68105a90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009900 tx=0x7f3c5802bc30 comp rx=0 tx=0).stop 2024-04-21T10:41:39.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.534+0000 7f3c6fe81640 1 -- 172.21.15.26:0/657038689 shutdown_connections 2024-04-21T10:41:39.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.534+0000 7f3c6fe81640 1 --2- 172.21.15.26:0/657038689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c68106170 0x7f3c6810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.534+0000 7f3c6fe81640 1 --2- 172.21.15.26:0/657038689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c68101a50 0x7f3c68105a90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.534+0000 7f3c6fe81640 1 --2- 172.21.15.26:0/657038689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c681010a0 0x7f3c68101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.534+0000 7f3c6fe81640 1 -- 172.21.15.26:0/657038689 >> 172.21.15.26:0/657038689 conn(0x7f3c68077eb0 msgr2=0x7f3c680782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:39.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.534+0000 7f3c6fe81640 1 -- 172.21.15.26:0/657038689 shutdown_connections 2024-04-21T10:41:39.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.534+0000 7f3c6fe81640 1 -- 172.21.15.26:0/657038689 wait complete. 2024-04-21T10:41:39.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6fe81640 1 Processor -- start 2024-04-21T10:41:39.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6fe81640 1 -- start start 2024-04-21T10:41:39.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6fe81640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c681010a0 0x7f3c68197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6fe81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c68106170 0x7f3c68197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6fe81640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c6819bed0 0x7f3c6819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6fe81640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3c68111090 con 0x7f3c68106170 2024-04-21T10:41:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6fe81640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3c68110f10 con 0x7f3c681010a0 2024-04-21T10:41:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6fe81640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3c68111210 con 0x7f3c6819bed0 2024-04-21T10:41:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6d41c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c68106170 0x7f3c68197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:39.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6d41c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c68106170 0x7f3c68197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55456/0 (socket says 172.21.15.26:55456) 2024-04-21T10:41:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6d41c640 1 -- 172.21.15.26:0/3212634110 learned_addr learned my addr 172.21.15.26:0/3212634110 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6e41e640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c6819bed0 0x7f3c6819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.535+0000 7f3c6dc1d640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c681010a0 0x7f3c68197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.536+0000 7f3c6d41c640 1 -- 172.21.15.26:0/3212634110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c6819bed0 msgr2=0x7f3c6819c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.536+0000 7f3c6d41c640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c6819bed0 0x7f3c6819c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.536+0000 7f3c6d41c640 1 -- 172.21.15.26:0/3212634110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c681010a0 msgr2=0x7f3c68197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.536+0000 7f3c6d41c640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c681010a0 0x7f3c68197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.536+0000 7f3c6d41c640 1 -- 172.21.15.26:0/3212634110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6819c9c0 con 0x7f3c68106170 2024-04-21T10:41:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.536+0000 7f3c6dc1d640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c681010a0 0x7f3c68197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:41:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.536+0000 7f3c6d41c640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c68106170 0x7f3c68197b50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3c58036040 tx=0x7f3c580045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.536+0000 7f3c5effd640 1 -- 172.21.15.26:0/3212634110 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c58043020 con 0x7f3c68106170 2024-04-21T10:41:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.536+0000 7f3c6fe81640 1 -- 172.21.15.26:0/3212634110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6819cc50 con 0x7f3c68106170 2024-04-21T10:41:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.536+0000 7f3c5effd640 1 -- 172.21.15.26:0/3212634110 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c5802ca70 con 0x7f3c68106170 2024-04-21T10:41:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.536+0000 7f3c5effd640 1 -- 172.21.15.26:0/3212634110 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c58003ac0 con 0x7f3c68106170 2024-04-21T10:41:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.536+0000 7f3c6fe81640 1 -- 172.21.15.26:0/3212634110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c681a4450 con 0x7f3c68106170 2024-04-21T10:41:39.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.537+0000 7f3c6fe81640 1 -- 172.21.15.26:0/3212634110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c680691c0 con 0x7f3c68106170 2024-04-21T10:41:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.538+0000 7f3c5effd640 1 -- 172.21.15.26:0/3212634110 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3c58046020 con 0x7f3c68106170 2024-04-21T10:41:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.539+0000 7f3c5effd640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f3c540a3d20 0x7f3c540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.539+0000 7f3c6dc1d640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f3c540a3d20 0x7f3c540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:39.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.539+0000 7f3c5effd640 1 -- 172.21.15.26:0/3212634110 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7f3c580f28a0 con 0x7f3c68106170 2024-04-21T10:41:39.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.539+0000 7f3c6dc1d640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f3c540a3d20 0x7f3c540a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009b50 tx=0x7f3c64005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:39.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.543+0000 7f3c5effd640 1 -- 172.21.15.26:0/3212634110 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c5804d020 con 0x7f3c68106170 2024-04-21T10:41:39.684 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:39 smithi026 ceph-mon[27965]: pgmap v333: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:41:39.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:39 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1617067736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:41:39.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:39 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3043129266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:41:39.685 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:39 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2463790993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:41:39.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.742+0000 7f3c6fe81640 1 -- 172.21.15.26:0/3212634110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3c68101a50 con 0x7f3c68106170 2024-04-21T10:41:39.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.743+0000 7f3c5effd640 1 -- 172.21.15.26:0/3212634110 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+940 (secure 0 0 0) 0x7f3c5802cc20 con 0x7f3c68106170 2024-04-21T10:41:39.743 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3802401516 2024-04-22T10:30:57.800969+0000 2024-04-21T10:41:39.743 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3125048023 2024-04-22T10:30:57.800969+0000 2024-04-21T10:41:39.743 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/4245538524 2024-04-22T10:30:57.800969+0000 2024-04-21T10:41:39.743 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2797499963 2024-04-22T10:30:57.800969+0000 2024-04-21T10:41:39.743 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/72610606 2024-04-22T10:30:57.800969+0000 2024-04-21T10:41:39.743 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2138811732 2024-04-22T10:22:02.083151+0000 2024-04-21T10:41:39.744 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/72610606 2024-04-22T10:30:57.800969+0000 2024-04-21T10:41:39.744 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/4216306847 2024-04-22T10:22:02.083151+0000 2024-04-21T10:41:39.744 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/878369551 2024-04-22T10:22:02.083151+0000 2024-04-21T10:41:39.744 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2916501364 2024-04-22T10:22:02.083151+0000 2024-04-21T10:41:39.744 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/3874640691 2024-04-22T10:21:35.999459+0000 2024-04-21T10:41:39.744 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/878369551 2024-04-22T10:22:02.083151+0000 2024-04-21T10:41:39.744 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3866216235 2024-04-22T10:21:35.999459+0000 2024-04-21T10:41:39.744 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/3874640691 2024-04-22T10:21:35.999459+0000 2024-04-21T10:41:39.745 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/1439453622 2024-04-22T10:21:35.999459+0000 2024-04-21T10:41:39.745 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3848404931 2024-04-22T10:21:35.999459+0000 2024-04-21T10:41:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:listed 16 entries 2024-04-21T10:41:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.745+0000 7f3c6fe81640 1 -- 172.21.15.26:0/3212634110 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f3c540a3d20 msgr2=0x7f3c540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.745+0000 7f3c6fe81640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f3c540a3d20 0x7f3c540a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009b50 tx=0x7f3c64005f90 comp rx=0 tx=0).stop 2024-04-21T10:41:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.745+0000 7f3c6fe81640 1 -- 172.21.15.26:0/3212634110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c68106170 msgr2=0x7f3c68197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.745+0000 7f3c6fe81640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c68106170 0x7f3c68197b50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3c58036040 tx=0x7f3c580045e0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.745+0000 7f3c6fe81640 1 -- 172.21.15.26:0/3212634110 shutdown_connections 2024-04-21T10:41:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.745+0000 7f3c6fe81640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f3c540a3d20 0x7f3c540a61e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.745+0000 7f3c6fe81640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3c6819bed0 0x7f3c6819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.745+0000 7f3c6fe81640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c68106170 0x7f3c68197b50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.745+0000 7f3c6fe81640 1 --2- 172.21.15.26:0/3212634110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c681010a0 0x7f3c68197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.745+0000 7f3c6fe81640 1 -- 172.21.15.26:0/3212634110 >> 172.21.15.26:0/3212634110 conn(0x7f3c68077eb0 msgr2=0x7f3c680ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.745+0000 7f3c6fe81640 1 -- 172.21.15.26:0/3212634110 shutdown_connections 2024-04-21T10:41:39.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:39.745+0000 7f3c6fe81640 1 -- 172.21.15.26:0/3212634110 wait complete. 2024-04-21T10:41:39.755 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-21T10:41:39.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:41:39.755 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-21T10:41:39.856 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-04-21T10:41:39.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:41:39.857 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-21T10:41:39.864 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:39 smithi190 ceph-mon[31423]: pgmap v333: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:41:39.864 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:39 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1617067736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:41:39.864 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:39 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3043129266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:41:39.865 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:39 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2463790993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:41:39.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:41:39.879 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-21T10:41:39.935 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-21T10:41:39.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:41:39.936 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-21T10:41:39.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:39 smithi169 ceph-mon[31469]: pgmap v333: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-21T10:41:39.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:39 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1617067736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T10:41:39.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:39 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3043129266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T10:41:39.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:39 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2463790993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T10:41:40.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T10:41:40.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-04-21T10:41:40.098 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-21T10:41:40.109 INFO:tasks.cephadm:Teardown begin 2024-04-21T10:41:40.109 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-21T10:41:40.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:41:40.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.214+0000 7f3a0b9db640 1 -- 172.21.15.26:0/3559205578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a0410c5a0 msgr2=0x7f3a0410e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.214+0000 7f3a0b9db640 1 --2- 172.21.15.26:0/3559205578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a0410c5a0 0x7f3a0410e9d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009ca0 tx=0x7f39fc02c180 comp rx=0 tx=0).stop 2024-04-21T10:41:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.214+0000 7f3a0b9db640 1 -- 172.21.15.26:0/3559205578 shutdown_connections 2024-04-21T10:41:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.214+0000 7f3a0b9db640 1 --2- 172.21.15.26:0/3559205578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a0410c5a0 0x7f3a0410e9d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.214+0000 7f3a0b9db640 1 --2- 172.21.15.26:0/3559205578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a04103af0 0x7f3a0410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.214+0000 7f3a0b9db640 1 --2- 172.21.15.26:0/3559205578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a041031d0 0x7f3a041035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.214+0000 7f3a0b9db640 1 -- 172.21.15.26:0/3559205578 >> 172.21.15.26:0/3559205578 conn(0x7f3a040771d0 msgr2=0x7f3a040775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.214+0000 7f3a0b9db640 1 -- 172.21.15.26:0/3559205578 shutdown_connections 2024-04-21T10:41:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.214+0000 7f3a0b9db640 1 -- 172.21.15.26:0/3559205578 wait complete. 2024-04-21T10:41:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a0b9db640 1 Processor -- start 2024-04-21T10:41:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a0b9db640 1 -- start start 2024-04-21T10:41:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a0b9db640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a041031d0 0x7f3a041986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a0b9db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a04103af0 0x7f3a04198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a0b9db640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a0410c5a0 0x7f3a0419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a0b9db640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041121a0 con 0x7f3a04103af0 2024-04-21T10:41:40.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a0b9db640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04112020 con 0x7f3a0410c5a0 2024-04-21T10:41:40.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a0b9db640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04112320 con 0x7f3a041031d0 2024-04-21T10:41:40.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a08f76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a04103af0 0x7f3a04198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:40.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a08f76640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a04103af0 0x7f3a04198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55476/0 (socket says 172.21.15.26:55476) 2024-04-21T10:41:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a08f76640 1 -- 172.21.15.26:0/2553292859 learned_addr learned my addr 172.21.15.26:0/2553292859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a09777640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a041031d0 0x7f3a041986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a09f78640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a0410c5a0 0x7f3a0419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.215+0000 7f3a08f76640 1 -- 172.21.15.26:0/2553292859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a041031d0 msgr2=0x7f3a041986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.216+0000 7f3a08f76640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a041031d0 0x7f3a041986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.216+0000 7f3a08f76640 1 -- 172.21.15.26:0/2553292859 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a0410c5a0 msgr2=0x7f3a0419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.216+0000 7f3a08f76640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a0410c5a0 0x7f3a0419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.216+0000 7f3a08f76640 1 -- 172.21.15.26:0/2553292859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0419d690 con 0x7f3a04103af0 2024-04-21T10:41:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.216+0000 7f3a08f76640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a04103af0 0x7f3a04198be0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f39f400e9f0 tx=0x7f39f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.216+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2553292859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f400cd80 con 0x7f3a04103af0 2024-04-21T10:41:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.216+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2553292859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39f40108e0 con 0x7f3a04103af0 2024-04-21T10:41:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.216+0000 7f3a0b9db640 1 -- 172.21.15.26:0/2553292859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0419d980 con 0x7f3a04103af0 2024-04-21T10:41:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.216+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2553292859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4018980 con 0x7f3a04103af0 2024-04-21T10:41:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.216+0000 7f3a0b9db640 1 -- 172.21.15.26:0/2553292859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a041a5210 con 0x7f3a04103af0 2024-04-21T10:41:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.217+0000 7f3a0b9db640 1 -- 172.21.15.26:0/2553292859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a04069230 con 0x7f3a04103af0 2024-04-21T10:41:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.218+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2553292859 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f39f4010430 con 0x7f3a04103af0 2024-04-21T10:41:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.219+0000 7f39fa7fc640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f39f00a3da0 0x7f39f00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.219+0000 7f3a09777640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f39f00a3da0 0x7f39f00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.219+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2553292859 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7f39f40d0f40 con 0x7f3a04103af0 2024-04-21T10:41:40.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.219+0000 7f3a09777640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f39f00a3da0 0x7f39f00a6260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009bd0 tx=0x7f3a00005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:40.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.223+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2553292859 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f39f409d5c0 con 0x7f3a04103af0 2024-04-21T10:41:40.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.450+0000 7f3a0b9db640 1 -- 172.21.15.26:0/2553292859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a041999f0 con 0x7f3a04103af0 2024-04-21T10:41:40.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.451+0000 7f39fa7fc640 1 -- 172.21.15.26:0/2553292859 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11884 (secure 0 0 0) 0x7f3a041999f0 con 0x7f3a04103af0 2024-04-21T10:41:40.450 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:41:40.450 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:35:27.498421+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24478,24488]},"id":1}]} 2024-04-21T10:41:40.451 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2024-04-21T10:41:40.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.453+0000 7f3a0b9db640 1 -- 172.21.15.26:0/2553292859 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f39f00a3da0 msgr2=0x7f39f00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.453+0000 7f3a0b9db640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f39f00a3da0 0x7f39f00a6260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009bd0 tx=0x7f3a00005f90 comp rx=0 tx=0).stop 2024-04-21T10:41:40.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.453+0000 7f3a0b9db640 1 -- 172.21.15.26:0/2553292859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a04103af0 msgr2=0x7f3a04198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.453+0000 7f3a0b9db640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a04103af0 0x7f3a04198be0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f39f400e9f0 tx=0x7f39f400eeb0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.453+0000 7f3a0b9db640 1 -- 172.21.15.26:0/2553292859 shutdown_connections 2024-04-21T10:41:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.453+0000 7f3a0b9db640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f39f00a3da0 0x7f39f00a6260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.453+0000 7f3a0b9db640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a0410c5a0 0x7f3a0419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.453+0000 7f3a0b9db640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a04103af0 0x7f3a04198be0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.453+0000 7f3a0b9db640 1 --2- 172.21.15.26:0/2553292859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a041031d0 0x7f3a041986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.453+0000 7f3a0b9db640 1 -- 172.21.15.26:0/2553292859 >> 172.21.15.26:0/2553292859 conn(0x7f3a040771d0 msgr2=0x7f3a040fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.453+0000 7f3a0b9db640 1 -- 172.21.15.26:0/2553292859 shutdown_connections 2024-04-21T10:41:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.453+0000 7f3a0b9db640 1 -- 172.21.15.26:0/2553292859 wait complete. 2024-04-21T10:41:40.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T10:41:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.566+0000 7f9eb12c1640 1 -- 172.21.15.26:0/2739713908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac069aa0 msgr2=0x7f9eac10ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.566+0000 7f9eb12c1640 1 --2- 172.21.15.26:0/2739713908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac069aa0 0x7f9eac10ede0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009ca0 tx=0x7f9ea402c0f0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.566+0000 7f9eb12c1640 1 -- 172.21.15.26:0/2739713908 shutdown_connections 2024-04-21T10:41:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.566+0000 7f9eb12c1640 1 --2- 172.21.15.26:0/2739713908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac069aa0 0x7f9eac10ede0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.566+0000 7f9eb12c1640 1 --2- 172.21.15.26:0/2739713908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eac0690e0 0x7f9eac069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.566+0000 7f9eb12c1640 1 --2- 172.21.15.26:0/2739713908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9eac106f50 0x7f9eac107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.566+0000 7f9eb12c1640 1 -- 172.21.15.26:0/2739713908 >> 172.21.15.26:0/2739713908 conn(0x7f9eac077e30 msgr2=0x7f9eac078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.567+0000 7f9eb12c1640 1 -- 172.21.15.26:0/2739713908 shutdown_connections 2024-04-21T10:41:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.567+0000 7f9eb12c1640 1 -- 172.21.15.26:0/2739713908 wait complete. 2024-04-21T10:41:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.567+0000 7f9eb12c1640 1 Processor -- start 2024-04-21T10:41:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.567+0000 7f9eb12c1640 1 -- start start 2024-04-21T10:41:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.567+0000 7f9eb12c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eac0690e0 0x7f9eac198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.567+0000 7f9eb12c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9eac069aa0 0x7f9eac198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eb12c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac106f50 0x7f9eac19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eb12c1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac112170 con 0x7f9eac106f50 2024-04-21T10:41:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eb12c1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac111ff0 con 0x7f9eac0690e0 2024-04-21T10:41:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eb12c1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1122f0 con 0x7f9eac069aa0 2024-04-21T10:41:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac106f50 0x7f9eac19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac106f50 0x7f9eac19cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55506/0 (socket says 172.21.15.26:55506) 2024-04-21T10:41:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eab7fe640 1 -- 172.21.15.26:0/3576296825 learned_addr learned my addr 172.21.15.26:0/3576296825 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eaaffd640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eac0690e0 0x7f9eac198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eaa7fc640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9eac069aa0 0x7f9eac198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eab7fe640 1 -- 172.21.15.26:0/3576296825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9eac069aa0 msgr2=0x7f9eac198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eab7fe640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9eac069aa0 0x7f9eac198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eab7fe640 1 -- 172.21.15.26:0/3576296825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eac0690e0 msgr2=0x7f9eac198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eab7fe640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eac0690e0 0x7f9eac198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eab7fe640 1 -- 172.21.15.26:0/3576296825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac19d5f0 con 0x7f9eac106f50 2024-04-21T10:41:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eab7fe640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac106f50 0x7f9eac19cec0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9ea402c700 tx=0x7f9ea40346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9e93fff640 1 -- 172.21.15.26:0/3576296825 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4041020 con 0x7f9eac106f50 2024-04-21T10:41:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eb12c1640 1 -- 172.21.15.26:0/3576296825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac19d880 con 0x7f9eac106f50 2024-04-21T10:41:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9e93fff640 1 -- 172.21.15.26:0/3576296825 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea4032960 con 0x7f9eac106f50 2024-04-21T10:41:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9e93fff640 1 -- 172.21.15.26:0/3576296825 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4039040 con 0x7f9eac106f50 2024-04-21T10:41:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.568+0000 7f9eb12c1640 1 -- 172.21.15.26:0/3576296825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac1a5170 con 0x7f9eac106f50 2024-04-21T10:41:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.569+0000 7f9eb12c1640 1 -- 172.21.15.26:0/3576296825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eac068de0 con 0x7f9eac106f50 2024-04-21T10:41:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.571+0000 7f9e93fff640 1 -- 172.21.15.26:0/3576296825 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9ea4044020 con 0x7f9eac106f50 2024-04-21T10:41:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.571+0000 7f9e93fff640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f9e940a3d20 0x7f9e940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.571+0000 7f9e93fff640 1 -- 172.21.15.26:0/3576296825 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7f9ea40f28e0 con 0x7f9eac106f50 2024-04-21T10:41:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.571+0000 7f9eaaffd640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f9e940a3d20 0x7f9e940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.572+0000 7f9eaaffd640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f9e940a3d20 0x7f9e940a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005fd0 tx=0x7f9e9c00b390 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.575+0000 7f9e93fff640 1 -- 172.21.15.26:0/3576296825 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ea404b020 con 0x7f9eac106f50 2024-04-21T10:41:40.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:40 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3212634110' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T10:41:40.776 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:40 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2553292859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:41:40.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.802+0000 7f9eb12c1640 1 -- 172.21.15.26:0/3576296825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9eac10a3f0 con 0x7f9eac106f50 2024-04-21T10:41:40.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.803+0000 7f9e93fff640 1 -- 172.21.15.26:0/3576296825 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11884 (secure 0 0 0) 0x7f9ea40bf190 con 0x7f9eac106f50 2024-04-21T10:41:40.803 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:41:40.803 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2165061396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2165061396},{"type":"v1","addr":"172.21.15.26:6835","nonce":2165061396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/4049177866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":4049177866},{"type":"v1","addr":"172.21.15.26:6837","nonce":4049177866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/3108850523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":3108850523},{"type":"v1","addr":"172.21.15.26:6839","nonce":3108850523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/1075023600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":1075023600},{"type":"v1","addr":"172.21.15.26:6841","nonce":1075023600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2779252400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2779252400},{"type":"v1","addr":"172.21.15.169:6833","nonce":2779252400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2081925034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2081925034},{"type":"v1","addr":"172.21.15.190:6833","nonce":2081925034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T10:29:01.278914+0000","modified":"2024-04-21T10:35:27.498421+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24496,"mds_1":24488,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6835/1594119566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1594119566},{"type":"v1","addr":"172.21.15.169:6835","nonce":1594119566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6837/2325983283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":2325983283},{"type":"v1","addr":"172.21.15.169:6837","nonce":2325983283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6835/1573483809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1573483809},{"type":"v1","addr":"172.21.15.190:6835","nonce":1573483809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6837/2091448511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":2091448511},{"type":"v1","addr":"172.21.15.190:6837","nonce":2091448511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":1,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6839/2428626465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":2428626465},{"type":"v1","addr":"172.21.15.169:6839","nonce":2428626465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/1574521545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":1574521545},{"type":"v1","addr":"172.21.15.190:6839","nonce":1574521545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24496,"qdb_cluster":[24496,24478,24488]},"id":1}]} 2024-04-21T10:41:40.803 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2024-04-21T10:41:40.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.805+0000 7f9eb12c1640 1 -- 172.21.15.26:0/3576296825 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f9e940a3d20 msgr2=0x7f9e940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.805+0000 7f9eb12c1640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f9e940a3d20 0x7f9e940a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005fd0 tx=0x7f9e9c00b390 comp rx=0 tx=0).stop 2024-04-21T10:41:40.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.805+0000 7f9eb12c1640 1 -- 172.21.15.26:0/3576296825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac106f50 msgr2=0x7f9eac19cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.805+0000 7f9eb12c1640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac106f50 0x7f9eac19cec0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9ea402c700 tx=0x7f9ea40346d0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.806+0000 7f9eb12c1640 1 -- 172.21.15.26:0/3576296825 shutdown_connections 2024-04-21T10:41:40.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.806+0000 7f9eb12c1640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f9e940a3d20 0x7f9e940a61e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.806+0000 7f9eb12c1640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac106f50 0x7f9eac19cec0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.806+0000 7f9eb12c1640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9eac069aa0 0x7f9eac198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.806+0000 7f9eb12c1640 1 --2- 172.21.15.26:0/3576296825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eac0690e0 0x7f9eac198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.806+0000 7f9eb12c1640 1 -- 172.21.15.26:0/3576296825 >> 172.21.15.26:0/3576296825 conn(0x7f9eac077e30 msgr2=0x7f9eac0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:40.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.806+0000 7f9eb12c1640 1 -- 172.21.15.26:0/3576296825 shutdown_connections 2024-04-21T10:41:40.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.806+0000 7f9eb12c1640 1 -- 172.21.15.26:0/3576296825 wait complete. 2024-04-21T10:41:40.815 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:40 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3212634110' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T10:41:40.815 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:40 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2553292859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:41:40.816 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T10:41:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 -- 172.21.15.26:0/4042542156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb540102190 msgr2=0x7fb540102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 --2- 172.21.15.26:0/4042542156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb540102190 0x7fb540102570 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009ca0 tx=0x7fb53c02c0f0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 -- 172.21.15.26:0/4042542156 shutdown_connections 2024-04-21T10:41:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 --2- 172.21.15.26:0/4042542156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb540107260 0x7fb54010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 --2- 172.21.15.26:0/4042542156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb540102b40 0x7fb540106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 --2- 172.21.15.26:0/4042542156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb540102190 0x7fb540102570 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 -- 172.21.15.26:0/4042542156 >> 172.21.15.26:0/4042542156 conn(0x7fb5400793e0 msgr2=0x7fb5400797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 -- 172.21.15.26:0/4042542156 shutdown_connections 2024-04-21T10:41:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 -- 172.21.15.26:0/4042542156 wait complete. 2024-04-21T10:41:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 Processor -- start 2024-04-21T10:41:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 -- start start 2024-04-21T10:41:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb540102190 0x7fb5401986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb540107260 0x7fb540198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.919+0000 7fb548724640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb54019cfa0 0x7fb54019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb548724640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb5401121a0 con 0x7fb540107260 2024-04-21T10:41:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb548724640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb540112020 con 0x7fb540102190 2024-04-21T10:41:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb548724640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb540112320 con 0x7fb54019cfa0 2024-04-21T10:41:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb545cbf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb540107260 0x7fb540198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb545cbf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb540107260 0x7fb540198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55532/0 (socket says 172.21.15.26:55532) 2024-04-21T10:41:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb545cbf640 1 -- 172.21.15.26:0/2109144669 learned_addr learned my addr 172.21.15.26:0/2109144669 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb5464c0640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb540102190 0x7fb5401986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb546cc1640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb54019cfa0 0x7fb54019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb545cbf640 1 -- 172.21.15.26:0/2109144669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb54019cfa0 msgr2=0x7fb54019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb545cbf640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb54019cfa0 0x7fb54019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb545cbf640 1 -- 172.21.15.26:0/2109144669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb540102190 msgr2=0x7fb5401986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb545cbf640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb540102190 0x7fb5401986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb545cbf640 1 -- 172.21.15.26:0/2109144669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54019da90 con 0x7fb540107260 2024-04-21T10:41:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb546cc1640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb54019cfa0 0x7fb54019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:41:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb545cbf640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb540107260 0x7fb540198c20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb53000d770 tx=0x7fb53000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb5377fe640 1 -- 172.21.15.26:0/2109144669 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530017070 con 0x7fb540107260 2024-04-21T10:41:40.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb5377fe640 1 -- 172.21.15.26:0/2109144669 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb530009c30 con 0x7fb540107260 2024-04-21T10:41:40.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb5377fe640 1 -- 172.21.15.26:0/2109144669 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53000f620 con 0x7fb540107260 2024-04-21T10:41:40.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb548724640 1 -- 172.21.15.26:0/2109144669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54019dd80 con 0x7fb540107260 2024-04-21T10:41:40.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.920+0000 7fb548724640 1 -- 172.21.15.26:0/2109144669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5401a5570 con 0x7fb540107260 2024-04-21T10:41:40.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.921+0000 7fb548724640 1 -- 172.21.15.26:0/2109144669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb540068de0 con 0x7fb540107260 2024-04-21T10:41:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.923+0000 7fb5377fe640 1 -- 172.21.15.26:0/2109144669 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb53001a070 con 0x7fb540107260 2024-04-21T10:41:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.923+0000 7fb5377fe640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fb52c0a3d20 0x7fb52c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.923+0000 7fb5464c0640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fb52c0a3d20 0x7fb52c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.923+0000 7fb5377fe640 1 -- 172.21.15.26:0/2109144669 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7fb5300d0440 con 0x7fb540107260 2024-04-21T10:41:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.924+0000 7fb5464c0640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fb52c0a3d20 0x7fb52c0a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb53c000f80 tx=0x7fb53c005c30 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:40.927+0000 7fb5377fe640 1 -- 172.21.15.26:0/2109144669 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb530009de0 con 0x7fb540107260 2024-04-21T10:41:40.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:40 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3212634110' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T10:41:40.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:40 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2553292859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:41:41.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.115+0000 7fb548724640 1 -- 172.21.15.26:0/2109144669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb540102b40 con 0x7fb540107260 2024-04-21T10:41:41.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.117+0000 7fb5377fe640 1 -- 172.21.15.26:0/2109144669 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23170 (secure 0 0 0) 0x7fb53009ca60 con 0x7fb540107260 2024-04-21T10:41:41.117 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-21T10:41:41.117 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":100,"fsid":"b77245c8-ffc8-11ee-bc93-c7b262605968","created":"2024-04-21T10:21:08.637646+0000","modified":"2024-04-21T10:41:13.960179+0000","last_up_change":"2024-04-21T10:28:11.692452+0000","last_in_change":"2024-04-21T10:27:51.765985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T10:24:28.176891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T10:28:59.417687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T10:29:00.234808+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6a8bc52a-346f-4a31-aad9-535e551ba41b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6803","nonce":3221939108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6805","nonce":3221939108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6809","nonce":3221939108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3221939108},{"type":"v1","addr":"172.21.15.26:6807","nonce":3221939108}]},"public_addr":"172.21.15.26:6803/3221939108","cluster_addr":"172.21.15.26:6805/3221939108","heartbeat_back_addr":"172.21.15.26:6809/3221939108","heartbeat_front_addr":"172.21.15.26:6807/3221939108","state":["exists","up"]},{"osd":1,"uuid":"ef42478b-fb1c-4dbc-82ea-b9c3c3c93f9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6801","nonce":4294740313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6803","nonce":4294740313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6807","nonce":4294740313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":4294740313},{"type":"v1","addr":"172.21.15.169:6805","nonce":4294740313}]},"public_addr":"172.21.15.169:6801/4294740313","cluster_addr":"172.21.15.169:6803/4294740313","heartbeat_back_addr":"172.21.15.169:6807/4294740313","heartbeat_front_addr":"172.21.15.169:6805/4294740313","state":["exists","up"]},{"osd":2,"uuid":"e7906664-6ec7-46e9-a500-50bac1bc7674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6801","nonce":2878992539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6803","nonce":2878992539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6807","nonce":2878992539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":2878992539},{"type":"v1","addr":"172.21.15.190:6805","nonce":2878992539}]},"public_addr":"172.21.15.190:6801/2878992539","cluster_addr":"172.21.15.190:6803/2878992539","heartbeat_back_addr":"172.21.15.190:6807/2878992539","heartbeat_front_addr":"172.21.15.190:6805/2878992539","state":["exists","up"]},{"osd":3,"uuid":"62499ade-dc25-4d67-8e9f-5486c2298565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6811","nonce":2335886169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6813","nonce":2335886169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6817","nonce":2335886169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2335886169},{"type":"v1","addr":"172.21.15.26:6815","nonce":2335886169}]},"public_addr":"172.21.15.26:6811/2335886169","cluster_addr":"172.21.15.26:6813/2335886169","heartbeat_back_addr":"172.21.15.26:6817/2335886169","heartbeat_front_addr":"172.21.15.26:6815/2335886169","state":["exists","up"]},{"osd":4,"uuid":"a1b50fcf-cfcd-42a2-b246-eb9d736aa437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6809","nonce":1708888642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6811","nonce":1708888642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6815","nonce":1708888642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":1708888642},{"type":"v1","addr":"172.21.15.169:6813","nonce":1708888642}]},"public_addr":"172.21.15.169:6809/1708888642","cluster_addr":"172.21.15.169:6811/1708888642","heartbeat_back_addr":"172.21.15.169:6815/1708888642","heartbeat_front_addr":"172.21.15.169:6813/1708888642","state":["exists","up"]},{"osd":5,"uuid":"33e19659-9fe5-4fe5-92e4-167c782735b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6809","nonce":427467914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6811","nonce":427467914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6815","nonce":427467914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":427467914},{"type":"v1","addr":"172.21.15.190:6813","nonce":427467914}]},"public_addr":"172.21.15.190:6809/427467914","cluster_addr":"172.21.15.190:6811/427467914","heartbeat_back_addr":"172.21.15.190:6815/427467914","heartbeat_front_addr":"172.21.15.190:6813/427467914","state":["exists","up"]},{"osd":6,"uuid":"20cf2d5b-6cea-4f55-9b15-2feeea193919","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6819","nonce":4211294337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6821","nonce":4211294337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6825","nonce":4211294337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":4211294337},{"type":"v1","addr":"172.21.15.26:6823","nonce":4211294337}]},"public_addr":"172.21.15.26:6819/4211294337","cluster_addr":"172.21.15.26:6821/4211294337","heartbeat_back_addr":"172.21.15.26:6825/4211294337","heartbeat_front_addr":"172.21.15.26:6823/4211294337","state":["exists","up"]},{"osd":7,"uuid":"184da4b1-7943-425d-8e97-c89af799e656","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6817","nonce":16938185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6819","nonce":16938185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6823","nonce":16938185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":16938185},{"type":"v1","addr":"172.21.15.169:6821","nonce":16938185}]},"public_addr":"172.21.15.169:6817/16938185","cluster_addr":"172.21.15.169:6819/16938185","heartbeat_back_addr":"172.21.15.169:6823/16938185","heartbeat_front_addr":"172.21.15.169:6821/16938185","state":["exists","up"]},{"osd":8,"uuid":"8da5499d-cd87-4182-87f7-6caffa3f5329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6817","nonce":250015578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6819","nonce":250015578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6823","nonce":250015578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":250015578},{"type":"v1","addr":"172.21.15.190:6821","nonce":250015578}]},"public_addr":"172.21.15.190:6817/250015578","cluster_addr":"172.21.15.190:6819/250015578","heartbeat_back_addr":"172.21.15.190:6823/250015578","heartbeat_front_addr":"172.21.15.190:6821/250015578","state":["exists","up"]},{"osd":9,"uuid":"47e78d57-1332-4623-a4cf-dcd4d4af2945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6827","nonce":3856298247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6829","nonce":3856298247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6833","nonce":3856298247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3856298247},{"type":"v1","addr":"172.21.15.26:6831","nonce":3856298247}]},"public_addr":"172.21.15.26:6827/3856298247","cluster_addr":"172.21.15.26:6829/3856298247","heartbeat_back_addr":"172.21.15.26:6833/3856298247","heartbeat_front_addr":"172.21.15.26:6831/3856298247","state":["exists","up"]},{"osd":10,"uuid":"11ff534c-dd49-446d-b532-f124b6a70b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6825","nonce":3439107945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6827","nonce":3439107945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6831","nonce":3439107945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":3439107945},{"type":"v1","addr":"172.21.15.169:6829","nonce":3439107945}]},"public_addr":"172.21.15.169:6825/3439107945","cluster_addr":"172.21.15.169:6827/3439107945","heartbeat_back_addr":"172.21.15.169:6831/3439107945","heartbeat_front_addr":"172.21.15.169:6829/3439107945","state":["exists","up"]},{"osd":11,"uuid":"2ec556b8-6ec3-4f59-ae2d-3f8f63e9c7ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6825","nonce":1221661638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6827","nonce":1221661638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6831","nonce":1221661638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1221661638},{"type":"v1","addr":"172.21.15.190:6829","nonce":1221661638}]},"public_addr":"172.21.15.190:6825/1221661638","cluster_addr":"172.21.15.190:6827/1221661638","heartbeat_back_addr":"172.21.15.190:6831/1221661638","heartbeat_front_addr":"172.21.15.190:6829/1221661638","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:35.309317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:23:59.910938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:24.413333+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:24:49.148639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:13.459632+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:25:38.162210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:03.317554+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:28.240158+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:26:53.314034+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:19.122819+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:27:44.181138+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T10:28:09.503727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/3802401516":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/3125048023":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4245538524":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2797499963":"2024-04-22T10:30:57.800969+0000","172.21.15.26:6801/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/2138811732":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/72610606":"2024-04-22T10:30:57.800969+0000","172.21.15.26:0/4216306847":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6801/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/2916501364":"2024-04-22T10:22:02.083151+0000","172.21.15.26:6800/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6800/878369551":"2024-04-22T10:22:02.083151+0000","172.21.15.26:0/3866216235":"2024-04-22T10:21:35.999459+0000","172.21.15.26:6801/3874640691":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/1439453622":"2024-04-22T10:21:35.999459+0000","172.21.15.26:0/3848404931":"2024-04-22T10:21:35.999459+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T10:41:41.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.119+0000 7fb548724640 1 -- 172.21.15.26:0/2109144669 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fb52c0a3d20 msgr2=0x7fb52c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.119+0000 7fb548724640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fb52c0a3d20 0x7fb52c0a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb53c000f80 tx=0x7fb53c005c30 comp rx=0 tx=0).stop 2024-04-21T10:41:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.119+0000 7fb548724640 1 -- 172.21.15.26:0/2109144669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb540107260 msgr2=0x7fb540198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.119+0000 7fb548724640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb540107260 0x7fb540198c20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb53000d770 tx=0x7fb53000dc30 comp rx=0 tx=0).stop 2024-04-21T10:41:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.119+0000 7fb548724640 1 -- 172.21.15.26:0/2109144669 shutdown_connections 2024-04-21T10:41:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.119+0000 7fb548724640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7fb52c0a3d20 0x7fb52c0a61e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.119+0000 7fb548724640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb54019cfa0 0x7fb54019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.119+0000 7fb548724640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb540107260 0x7fb540198c20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.119+0000 7fb548724640 1 --2- 172.21.15.26:0/2109144669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb540102190 0x7fb5401986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.119+0000 7fb548724640 1 -- 172.21.15.26:0/2109144669 >> 172.21.15.26:0/2109144669 conn(0x7fb5400793e0 msgr2=0x7fb540101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:41.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.119+0000 7fb548724640 1 -- 172.21.15.26:0/2109144669 shutdown_connections 2024-04-21T10:41:41.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.119+0000 7fb548724640 1 -- 172.21.15.26:0/2109144669 wait complete. 2024-04-21T10:41:41.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-21T10:41:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.233+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2507708653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e88101a40 msgr2=0x7f6e88105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.233+0000 7f6e8cd46640 1 --2- 172.21.15.26:0/2507708653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e88101a40 0x7f6e88105a80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009900 tx=0x7f6e7c02bc30 comp rx=0 tx=0).stop 2024-04-21T10:41:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.233+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2507708653 shutdown_connections 2024-04-21T10:41:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.233+0000 7f6e8cd46640 1 --2- 172.21.15.26:0/2507708653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e88106160 0x7f6e8810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.233+0000 7f6e8cd46640 1 --2- 172.21.15.26:0/2507708653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e88101a40 0x7f6e88105a80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.233+0000 7f6e8cd46640 1 --2- 172.21.15.26:0/2507708653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e88101090 0x7f6e88101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.233+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2507708653 >> 172.21.15.26:0/2507708653 conn(0x7f6e88077f50 msgr2=0x7f6e88078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.233+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2507708653 shutdown_connections 2024-04-21T10:41:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.233+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2507708653 wait complete. 2024-04-21T10:41:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8cd46640 1 Processor -- start 2024-04-21T10:41:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8cd46640 1 -- start start 2024-04-21T10:41:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8cd46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e88101090 0x7f6e88198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8cd46640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e88106160 0x7f6e88198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8cd46640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e8819d020 0x7f6e8819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8cd46640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6e88112200 con 0x7f6e88101090 2024-04-21T10:41:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8cd46640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6e88112080 con 0x7f6e88106160 2024-04-21T10:41:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8cd46640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6e88112380 con 0x7f6e8819d020 2024-04-21T10:41:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e88101090 0x7f6e88198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e88101090 0x7f6e88198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55562/0 (socket says 172.21.15.26:55562) 2024-04-21T10:41:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8659c640 1 -- 172.21.15.26:0/2502429537 learned_addr learned my addr 172.21.15.26:0/2502429537 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e85d9b640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e88106160 0x7f6e88198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8659c640 1 -- 172.21.15.26:0/2502429537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e8819d020 msgr2=0x7f6e8819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8659c640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e8819d020 0x7f6e8819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8659c640 1 -- 172.21.15.26:0/2502429537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e88106160 msgr2=0x7f6e88198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8659c640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e88106160 0x7f6e88198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e8659c640 1 -- 172.21.15.26:0/2502429537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8819db10 con 0x7f6e88101090 2024-04-21T10:41:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.234+0000 7f6e85d9b640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e88106160 0x7f6e88198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T10:41:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.235+0000 7f6e8659c640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e88101090 0x7f6e88198760 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800c920 tx=0x7f6e7800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.235+0000 7f6e777fe640 1 -- 172.21.15.26:0/2502429537 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e78007c70 con 0x7f6e88101090 2024-04-21T10:41:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.235+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2502429537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8819de00 con 0x7f6e88101090 2024-04-21T10:41:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.235+0000 7f6e777fe640 1 -- 172.21.15.26:0/2502429537 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e78007e10 con 0x7f6e88101090 2024-04-21T10:41:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.235+0000 7f6e777fe640 1 -- 172.21.15.26:0/2502429537 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e78018820 con 0x7f6e88101090 2024-04-21T10:41:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.235+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2502429537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e881a55f0 con 0x7f6e88101090 2024-04-21T10:41:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.236+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2502429537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e88068d70 con 0x7f6e88101090 2024-04-21T10:41:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.237+0000 7f6e777fe640 1 -- 172.21.15.26:0/2502429537 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6e7800f450 con 0x7f6e88101090 2024-04-21T10:41:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.238+0000 7f6e777fe640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f6e700a3d20 0x7f6e700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.238+0000 7f6e85d9b640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f6e700a3d20 0x7f6e700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.238+0000 7f6e777fe640 1 -- 172.21.15.26:0/2502429537 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8550+0+0 (secure 0 0 0) 0x7f6e780d0c90 con 0x7f6e88101090 2024-04-21T10:41:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.238+0000 7f6e85d9b640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f6e700a3d20 0x7f6e700a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c036040 tx=0x7f6e7c000bc0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.241+0000 7f6e777fe640 1 -- 172.21.15.26:0/2502429537 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e78014030 con 0x7f6e88101090 2024-04-21T10:41:41.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.467+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2502429537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6e88199ab0 con 0x7f6e88101090 2024-04-21T10:41:41.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.469+0000 7f6e777fe640 1 -- 172.21.15.26:0/2502429537 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f6e7809d3a0 con 0x7f6e88101090 2024-04-21T10:41:41.469 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-21T10:41:41.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.472+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2502429537 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f6e700a3d20 msgr2=0x7f6e700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:41.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.472+0000 7f6e8cd46640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f6e700a3d20 0x7f6e700a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c036040 tx=0x7f6e7c000bc0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.472+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2502429537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e88101090 msgr2=0x7f6e88198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:41.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.472+0000 7f6e8cd46640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e88101090 0x7f6e88198760 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800c920 tx=0x7f6e7800cde0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.472+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2502429537 shutdown_connections 2024-04-21T10:41:41.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.472+0000 7f6e8cd46640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f6e700a3d20 0x7f6e700a61e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.472+0000 7f6e8cd46640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e8819d020 0x7f6e8819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.472+0000 7f6e8cd46640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e88106160 0x7f6e88198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.472+0000 7f6e8cd46640 1 --2- 172.21.15.26:0/2502429537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e88101090 0x7f6e88198760 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.472+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2502429537 >> 172.21.15.26:0/2502429537 conn(0x7f6e88077f50 msgr2=0x7f6e880fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:41.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.472+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2502429537 shutdown_connections 2024-04-21T10:41:41.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.472+0000 7f6e8cd46640 1 -- 172.21.15.26:0/2502429537 wait complete. 2024-04-21T10:41:41.483 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-21T10:41:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.591+0000 7f01889c7640 1 -- 172.21.15.26:0/2451309108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841021a0 msgr2=0x7f0184102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.591+0000 7f01889c7640 1 --2- 172.21.15.26:0/2451309108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841021a0 0x7f0184102580 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0174009900 tx=0x7f017402bc30 comp rx=0 tx=0).stop 2024-04-21T10:41:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.591+0000 7f01889c7640 1 -- 172.21.15.26:0/2451309108 shutdown_connections 2024-04-21T10:41:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.591+0000 7f01889c7640 1 --2- 172.21.15.26:0/2451309108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0184107270 0x7f018410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.591+0000 7f01889c7640 1 --2- 172.21.15.26:0/2451309108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0184102b50 0x7f0184106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.591+0000 7f01889c7640 1 --2- 172.21.15.26:0/2451309108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841021a0 0x7f0184102580 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.591+0000 7f01889c7640 1 -- 172.21.15.26:0/2451309108 >> 172.21.15.26:0/2451309108 conn(0x7f01840fda40 msgr2=0x7f01840ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.591+0000 7f01889c7640 1 -- 172.21.15.26:0/2451309108 shutdown_connections 2024-04-21T10:41:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.591+0000 7f01889c7640 1 -- 172.21.15.26:0/2451309108 wait complete. 2024-04-21T10:41:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f01889c7640 1 Processor -- start 2024-04-21T10:41:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f01889c7640 1 -- start start 2024-04-21T10:41:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f01889c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841021a0 0x7f01841987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f01889c7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0184107270 0x7f0184198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f01889c7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f018419d020 0x7f018419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f01889c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f01841121d0 con 0x7f01841021a0 2024-04-21T10:41:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f01889c7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0184112050 con 0x7f0184107270 2024-04-21T10:41:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f01889c7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0184112350 con 0x7f018419d020 2024-04-21T10:41:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f018259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841021a0 0x7f01841987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f018259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841021a0 0x7f01841987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55586/0 (socket says 172.21.15.26:55586) 2024-04-21T10:41:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f018259c640 1 -- 172.21.15.26:0/1990001438 learned_addr learned my addr 172.21.15.26:0/1990001438 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f0182d9d640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f018419d020 0x7f018419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.592+0000 7f0179d9b640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0184107270 0x7f0184198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f018259c640 1 -- 172.21.15.26:0/1990001438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f018419d020 msgr2=0x7f018419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f018259c640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f018419d020 0x7f018419d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f018259c640 1 -- 172.21.15.26:0/1990001438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0184107270 msgr2=0x7f0184198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f018259c640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0184107270 0x7f0184198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f018259c640 1 -- 172.21.15.26:0/1990001438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018419db10 con 0x7f01841021a0 2024-04-21T10:41:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f0182d9d640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f018419d020 0x7f018419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T10:41:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f0179d9b640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0184107270 0x7f0184198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T10:41:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f018259c640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841021a0 0x7f01841987b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0174005940 tx=0x7f017402c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f017bfff640 1 -- 172.21.15.26:0/1990001438 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174043020 con 0x7f01841021a0 2024-04-21T10:41:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f01889c7640 1 -- 172.21.15.26:0/1990001438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018419dda0 con 0x7f01841021a0 2024-04-21T10:41:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f017bfff640 1 -- 172.21.15.26:0/1990001438 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f017402ca70 con 0x7f01841021a0 2024-04-21T10:41:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f017bfff640 1 -- 172.21.15.26:0/1990001438 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01740039f0 con 0x7f01841021a0 2024-04-21T10:41:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.593+0000 7f01889c7640 1 -- 172.21.15.26:0/1990001438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01841a55c0 con 0x7f01841021a0 2024-04-21T10:41:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.594+0000 7f01889c7640 1 -- 172.21.15.26:0/1990001438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01840691c0 con 0x7f01841021a0 2024-04-21T10:41:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.595+0000 7f017bfff640 1 -- 172.21.15.26:0/1990001438 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0174046020 con 0x7f01841021a0 2024-04-21T10:41:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.596+0000 7f017bfff640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f01700a3d20 0x7f01700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.596+0000 7f017bfff640 1 -- 172.21.15.26:0/1990001438 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8764+0+0 (secure 0 0 0) 0x7f01740f2930 con 0x7f01841021a0 2024-04-21T10:41:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.596+0000 7f0179d9b640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f01700a3d20 0x7f01700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:41.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.597+0000 7f0179d9b640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f01700a3d20 0x7f01700a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f01840fdef0 tx=0x7f016c005f90 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.601+0000 7f017bfff640 1 -- 172.21.15.26:0/1990001438 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f017404d020 con 0x7f01841021a0 2024-04-21T10:41:41.776 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i[62749]: ignoring --setuser ceph since I am not root 2024-04-21T10:41:41.777 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i[62749]: ignoring --setgroup ceph since I am not root 2024-04-21T10:41:41.777 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-mds[62753]: -- 172.21.15.190:0/642796027 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cca5cd7a40 con 0x55cca5cab000 2024-04-21T10:41:41.777 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i[62749]: starting mds.i at 2024-04-21T10:41:41.777 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-f[62124]: ignoring --setuser ceph since I am not root 2024-04-21T10:41:41.778 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-f[62124]: ignoring --setgroup ceph since I am not root 2024-04-21T10:41:41.778 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-mds[62128]: -- 172.21.15.190:0/2395879216 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5578d9bdda40 con 0x5578d9bb1000 2024-04-21T10:41:41.778 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-f[62124]: starting mds.f at 2024-04-21T10:41:41.778 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c[62429]: ignoring --setuser ceph since I am not root 2024-04-21T10:41:41.778 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c[62429]: ignoring --setgroup ceph since I am not root 2024-04-21T10:41:41.778 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-mds[62433]: -- 172.21.15.190:0/1800463207 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56543ff13a40 con 0x56543fee7000 2024-04-21T10:41:41.779 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c[62429]: starting mds.c at 2024-04-21T10:41:41.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-mon[31423]: pgmap v334: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:41:41.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3576296825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:41:41.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2109144669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:41:41.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2502429537' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T10:41:41.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-mon[31423]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T10:41:41.779 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-mon[31423]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T10:41:41.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/2502429537' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T10:41:41.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-mon[31423]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T10:41:41.780 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:41 smithi190 ceph-mon[31423]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-21T10:41:41.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:41.826+0000 7f01889c7640 1 -- 172.21.15.26:0/1990001438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f018410a500 con 0x7f01841021a0 2024-04-21T10:41:41.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:41 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a[27961]: 2024-04-21T10:41:41.468+0000 7f4e9c46a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T10:41:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:41 smithi026 ceph-mon[27965]: pgmap v334: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:41:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:41 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3576296825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:41:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:41 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2109144669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:41:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:41 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2502429537' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T10:41:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:41 smithi026 ceph-mon[27965]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T10:41:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:41 smithi026 ceph-mon[27965]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T10:41:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:41 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/2502429537' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T10:41:41.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:41 smithi026 ceph-mon[27965]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T10:41:41.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:41 smithi026 ceph-mon[27965]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-21T10:41:41.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-mon[31469]: pgmap v334: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T10:41:41.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3576296825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T10:41:41.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2109144669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T10:41:41.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2502429537' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T10:41:41.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-mon[31469]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T10:41:41.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-mon[31469]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T10:41:41.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/2502429537' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T10:41:41.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-mon[31469]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T10:41:41.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-mon[31469]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-21T10:41:41.949 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e[61511]: ignoring --setuser ceph since I am not root 2024-04-21T10:41:41.949 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e[61511]: ignoring --setgroup ceph since I am not root 2024-04-21T10:41:41.950 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-mds[61515]: -- 172.21.15.169:0/2929949603 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b3ba71a40 con 0x563b3ba44c00 2024-04-21T10:41:41.950 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e[61511]: starting mds.e at 2024-04-21T10:41:41.950 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-h[61820]: ignoring --setuser ceph since I am not root 2024-04-21T10:41:41.950 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-h[61820]: ignoring --setgroup ceph since I am not root 2024-04-21T10:41:41.950 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-mds[61824]: -- 172.21.15.169:0/2770902248 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5609ccf1fa40 con 0x5609ccef2c00 2024-04-21T10:41:41.951 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-h[61820]: starting mds.h at 2024-04-21T10:41:41.951 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b[62141]: ignoring --setuser ceph since I am not root 2024-04-21T10:41:41.951 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b[62141]: ignoring --setgroup ceph since I am not root 2024-04-21T10:41:41.951 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-mds[62145]: -- 172.21.15.169:0/361525157 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d12c37a40 con 0x556d12c0ac00 2024-04-21T10:41:41.951 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b[62141]: starting mds.b at 2024-04-21T10:41:41.951 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:41:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b[62141]: 2024-04-21T10:41:41.517+0000 7f7b60dd1700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2024-04-21T10:41:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.471+0000 7f017bfff640 1 -- 172.21.15.26:0/1990001438 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f017402cc20 con 0x7f01841021a0 2024-04-21T10:41:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.474+0000 7f01889c7640 1 -- 172.21.15.26:0/1990001438 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f01700a3d20 msgr2=0x7f01700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.474+0000 7f01889c7640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f01700a3d20 0x7f01700a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f01840fdef0 tx=0x7f016c005f90 comp rx=0 tx=0).stop 2024-04-21T10:41:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.474+0000 7f01889c7640 1 -- 172.21.15.26:0/1990001438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841021a0 msgr2=0x7f01841987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.474+0000 7f01889c7640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841021a0 0x7f01841987b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0174005940 tx=0x7f017402c5f0 comp rx=0 tx=0).stop 2024-04-21T10:41:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.474+0000 7f01889c7640 1 -- 172.21.15.26:0/1990001438 shutdown_connections 2024-04-21T10:41:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.474+0000 7f01889c7640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f01700a3d20 0x7f01700a61e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.474+0000 7f01889c7640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f018419d020 0x7f018419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.474+0000 7f01889c7640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0184107270 0x7f0184198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.474+0000 7f01889c7640 1 --2- 172.21.15.26:0/1990001438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01841021a0 0x7f01841987b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.474+0000 7f01889c7640 1 -- 172.21.15.26:0/1990001438 >> 172.21.15.26:0/1990001438 conn(0x7f01840fda40 msgr2=0x7f01840fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.474+0000 7f01889c7640 1 -- 172.21.15.26:0/1990001438 shutdown_connections 2024-04-21T10:41:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.474+0000 7f01889c7640 1 -- 172.21.15.26:0/1990001438 wait complete. 2024-04-21T10:41:42.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-21T10:41:42.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.604+0000 7f9fd8b59640 1 -- 172.21.15.26:0/483989212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4104d10 msgr2=0x7f9fd4108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:42.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.604+0000 7f9fd8b59640 1 --2- 172.21.15.26:0/483989212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4104d10 0x7f9fd4108d50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009900 tx=0x7f9fbc02bc30 comp rx=0 tx=0).stop 2024-04-21T10:41:42.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.604+0000 7f9fd8b59640 1 -- 172.21.15.26:0/483989212 shutdown_connections 2024-04-21T10:41:42.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.604+0000 7f9fd8b59640 1 --2- 172.21.15.26:0/483989212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd4109430 0x7f9fd4111000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:42.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.604+0000 7f9fd8b59640 1 --2- 172.21.15.26:0/483989212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4104d10 0x7f9fd4108d50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:42.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.604+0000 7f9fd8b59640 1 --2- 172.21.15.26:0/483989212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fd4104360 0x7f9fd4104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:42.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.604+0000 7f9fd8b59640 1 -- 172.21.15.26:0/483989212 >> 172.21.15.26:0/483989212 conn(0x7f9fd40ffc80 msgr2=0x7f9fd41020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:42.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.604+0000 7f9fd8b59640 1 -- 172.21.15.26:0/483989212 shutdown_connections 2024-04-21T10:41:42.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.604+0000 7f9fd8b59640 1 -- 172.21.15.26:0/483989212 wait complete. 2024-04-21T10:41:42.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.605+0000 7f9fd8b59640 1 Processor -- start 2024-04-21T10:41:42.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.605+0000 7f9fd8b59640 1 -- start start 2024-04-21T10:41:42.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.605+0000 7f9fd8b59640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fd4104360 0x7f9fd41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.605+0000 7f9fd8b59640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4109430 0x7f9fd41a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.605+0000 7f9fd8b59640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd419a730 0x7f9fd419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.605+0000 7f9fd8b59640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4114350 con 0x7f9fd4109430 2024-04-21T10:41:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.605+0000 7f9fd8b59640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd41141d0 con 0x7f9fd419a730 2024-04-21T10:41:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.605+0000 7f9fd8b59640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd41144d0 con 0x7f9fd4104360 2024-04-21T10:41:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fcbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4109430 0x7f9fd41a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fcbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4109430 0x7f9fd41a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55598/0 (socket says 172.21.15.26:55598) 2024-04-21T10:41:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fcbfff640 1 -- 172.21.15.26:0/3686316572 learned_addr learned my addr 172.21.15.26:0/3686316572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fd2d9d640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd419a730 0x7f9fd419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fd259c640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fd4104360 0x7f9fd41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fcbfff640 1 -- 172.21.15.26:0/3686316572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fd4104360 msgr2=0x7f9fd41a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fcbfff640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fd4104360 0x7f9fd41a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fcbfff640 1 -- 172.21.15.26:0/3686316572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd419a730 msgr2=0x7f9fd419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fcbfff640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd419a730 0x7f9fd419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fcbfff640 1 -- 172.21.15.26:0/3686316572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd419b330 con 0x7f9fd4109430 2024-04-21T10:41:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fcbfff640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4109430 0x7f9fd41a0bb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc036040 tx=0x7f9fbc02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fcb7fe640 1 -- 172.21.15.26:0/3686316572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc043020 con 0x7f9fd4109430 2024-04-21T10:41:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fcb7fe640 1 -- 172.21.15.26:0/3686316572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fbc02cbe0 con 0x7f9fd4109430 2024-04-21T10:41:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fd8b59640 1 -- 172.21.15.26:0/3686316572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd419b5c0 con 0x7f9fd4109430 2024-04-21T10:41:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fcb7fe640 1 -- 172.21.15.26:0/3686316572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc003bf0 con 0x7f9fd4109430 2024-04-21T10:41:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.606+0000 7f9fd8b59640 1 -- 172.21.15.26:0/3686316572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd41a7760 con 0x7f9fd4109430 2024-04-21T10:41:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.607+0000 7f9fd8b59640 1 -- 172.21.15.26:0/3686316572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd4105830 con 0x7f9fd4109430 2024-04-21T10:41:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.609+0000 7f9fcb7fe640 1 -- 172.21.15.26:0/3686316572 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9fbc046020 con 0x7f9fd4109430 2024-04-21T10:41:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.609+0000 7f9fcb7fe640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f9fc00a3d20 0x7f9fc00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:42.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.609+0000 7f9fd259c640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f9fc00a3d20 0x7f9fc00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:42.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.609+0000 7f9fcb7fe640 1 -- 172.21.15.26:0/3686316572 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8764+0+0 (secure 0 0 0) 0x7f9fbc0f2950 con 0x7f9fd4109430 2024-04-21T10:41:42.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.610+0000 7f9fd259c640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f9fc00a3d20 0x7f9fc00a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400bbe0 tx=0x7f9fc400b390 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:42.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.614+0000 7f9fcb7fe640 1 -- 172.21.15.26:0/3686316572 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9fbc04d020 con 0x7f9fd4109430 2024-04-21T10:41:42.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:42.801+0000 7f9fd8b59640 1 -- 172.21.15.26:0/3686316572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9fd4104d10 con 0x7f9fd4109430 2024-04-21T10:41:42.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1990001438' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T10:41:42.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T10:41:42.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T10:41:42.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: Cluster is now healthy 2024-04-21T10:41:42.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/1990001438' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T10:41:42.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.190:6834/1236546081,v1:172.21.15.190:6837/1236546081] up:boot 2024-04-21T10:41:42.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.190:6836/625901232,v1:172.21.15.190:6839/625901232] up:boot 2024-04-21T10:41:42.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.190:6835/3033173935,v1:172.21.15.190:6838/3033173935] up:boot 2024-04-21T10:41:42.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.169:6834/3921604256,v1:172.21.15.169:6837/3921604256] up:boot 2024-04-21T10:41:42.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.169:6835/2698082916,v1:172.21.15.169:6838/2698082916] up:boot 2024-04-21T10:41:42.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: mds.? [v2:172.21.15.169:6836/2106659241,v1:172.21.15.169:6839/2106659241] up:boot 2024-04-21T10:41:42.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: fsmap 12 up:standby 2024-04-21T10:41:42.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T10:41:42.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T10:41:42.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T10:41:42.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T10:41:42.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T10:41:42.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:42 smithi026 ceph-mon[27965]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T10:41:42.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1990001438' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T10:41:42.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T10:41:42.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T10:41:42.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: Cluster is now healthy 2024-04-21T10:41:42.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/1990001438' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T10:41:42.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.190:6834/1236546081,v1:172.21.15.190:6837/1236546081] up:boot 2024-04-21T10:41:42.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.190:6836/625901232,v1:172.21.15.190:6839/625901232] up:boot 2024-04-21T10:41:42.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.190:6835/3033173935,v1:172.21.15.190:6838/3033173935] up:boot 2024-04-21T10:41:42.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.169:6834/3921604256,v1:172.21.15.169:6837/3921604256] up:boot 2024-04-21T10:41:42.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.169:6835/2698082916,v1:172.21.15.169:6838/2698082916] up:boot 2024-04-21T10:41:42.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: mds.? [v2:172.21.15.169:6836/2106659241,v1:172.21.15.169:6839/2106659241] up:boot 2024-04-21T10:41:42.949 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: fsmap 12 up:standby 2024-04-21T10:41:42.950 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T10:41:42.950 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T10:41:42.950 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T10:41:42.950 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T10:41:42.950 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T10:41:42.950 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:42 smithi169 ceph-mon[31469]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T10:41:43.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1990001438' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T10:41:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T10:41:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T10:41:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: Cluster is now healthy 2024-04-21T10:41:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/1990001438' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T10:41:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.190:6834/1236546081,v1:172.21.15.190:6837/1236546081] up:boot 2024-04-21T10:41:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.190:6836/625901232,v1:172.21.15.190:6839/625901232] up:boot 2024-04-21T10:41:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.190:6835/3033173935,v1:172.21.15.190:6838/3033173935] up:boot 2024-04-21T10:41:43.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.169:6834/3921604256,v1:172.21.15.169:6837/3921604256] up:boot 2024-04-21T10:41:43.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.169:6835/2698082916,v1:172.21.15.169:6838/2698082916] up:boot 2024-04-21T10:41:43.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: mds.? [v2:172.21.15.169:6836/2106659241,v1:172.21.15.169:6839/2106659241] up:boot 2024-04-21T10:41:43.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: fsmap 12 up:standby 2024-04-21T10:41:43.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T10:41:43.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T10:41:43.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T10:41:43.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T10:41:43.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T10:41:43.028 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:42 smithi190 ceph-mon[31423]: from='mgr.24869 172.21.15.26:0/3143738463' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T10:41:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.568+0000 7f9fcb7fe640 1 -- 172.21.15.26:0/3686316572 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7f9fbc0bf010 con 0x7f9fd4109430 2024-04-21T10:41:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-04-21T10:41:43.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.571+0000 7f9fd8b59640 1 -- 172.21.15.26:0/3686316572 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f9fc00a3d20 msgr2=0x7f9fc00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:43.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.571+0000 7f9fd8b59640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f9fc00a3d20 0x7f9fc00a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400bbe0 tx=0x7f9fc400b390 comp rx=0 tx=0).stop 2024-04-21T10:41:43.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.571+0000 7f9fd8b59640 1 -- 172.21.15.26:0/3686316572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4109430 msgr2=0x7f9fd41a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:43.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.571+0000 7f9fd8b59640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4109430 0x7f9fd41a0bb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc036040 tx=0x7f9fbc02c7e0 comp rx=0 tx=0).stop 2024-04-21T10:41:43.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.571+0000 7f9fd8b59640 1 -- 172.21.15.26:0/3686316572 shutdown_connections 2024-04-21T10:41:43.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.571+0000 7f9fd8b59640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f9fc00a3d20 0x7f9fc00a61e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:43.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.571+0000 7f9fd8b59640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9fd419a730 0x7f9fd419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:43.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.571+0000 7f9fd8b59640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4109430 0x7f9fd41a0bb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:43.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.571+0000 7f9fd8b59640 1 --2- 172.21.15.26:0/3686316572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fd4104360 0x7f9fd41a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:43.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.571+0000 7f9fd8b59640 1 -- 172.21.15.26:0/3686316572 >> 172.21.15.26:0/3686316572 conn(0x7f9fd40ffc80 msgr2=0x7f9fd4101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:43.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.572+0000 7f9fd8b59640 1 -- 172.21.15.26:0/3686316572 shutdown_connections 2024-04-21T10:41:43.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.572+0000 7f9fd8b59640 1 -- 172.21.15.26:0/3686316572 wait complete. 2024-04-21T10:41:43.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-21T10:41:43.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.700+0000 7f108b411640 1 -- 172.21.15.26:0/4054111401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108406a840 msgr2=0x7f108406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:43.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.700+0000 7f108b411640 1 --2- 172.21.15.26:0/4054111401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108406a840 0x7f108406ac20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1080009900 tx=0x7f108002bc30 comp rx=0 tx=0).stop 2024-04-21T10:41:43.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.700+0000 7f108b411640 1 -- 172.21.15.26:0/4054111401 shutdown_connections 2024-04-21T10:41:43.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.700+0000 7f108b411640 1 --2- 172.21.15.26:0/4054111401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1084107270 0x7f108410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:43.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.700+0000 7f108b411640 1 --2- 172.21.15.26:0/4054111401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f108406b1f0 0x7f1084106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:43.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.700+0000 7f108b411640 1 --2- 172.21.15.26:0/4054111401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108406a840 0x7f108406ac20 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:43.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.700+0000 7f108b411640 1 -- 172.21.15.26:0/4054111401 >> 172.21.15.26:0/4054111401 conn(0x7f1084077eb0 msgr2=0x7f10840782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:43.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.700+0000 7f108b411640 1 -- 172.21.15.26:0/4054111401 shutdown_connections 2024-04-21T10:41:43.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.700+0000 7f108b411640 1 -- 172.21.15.26:0/4054111401 wait complete. 2024-04-21T10:41:43.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.701+0000 7f108b411640 1 Processor -- start 2024-04-21T10:41:43.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.701+0000 7f108b411640 1 -- start start 2024-04-21T10:41:43.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.701+0000 7f108b411640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108406a840 0x7f1084198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:43.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.701+0000 7f108b411640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1084107270 0x7f1084198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:43.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.701+0000 7f108b411640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f108419cff0 0x7f108419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:43.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.701+0000 7f108b411640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f10841121b0 con 0x7f1084107270 2024-04-21T10:41:43.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.701+0000 7f108b411640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1084112030 con 0x7f108419cff0 2024-04-21T10:41:43.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.701+0000 7f108b411640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1084112330 con 0x7f108406a840 2024-04-21T10:41:43.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.701+0000 7f10889ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1084107270 0x7f1084198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:43.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.701+0000 7f10889ac640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1084107270 0x7f1084198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55604/0 (socket says 172.21.15.26:55604) 2024-04-21T10:41:43.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.701+0000 7f10889ac640 1 -- 172.21.15.26:0/215492660 learned_addr learned my addr 172.21.15.26:0/215492660 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-21T10:41:43.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.701+0000 7f10899ae640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f108419cff0 0x7f108419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.702+0000 7f10891ad640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108406a840 0x7f1084198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.702+0000 7f10889ac640 1 -- 172.21.15.26:0/215492660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108406a840 msgr2=0x7f1084198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.702+0000 7f10889ac640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108406a840 0x7f1084198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.702+0000 7f10889ac640 1 -- 172.21.15.26:0/215492660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f108419cff0 msgr2=0x7f108419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.702+0000 7f10889ac640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f108419cff0 0x7f108419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.702+0000 7f10889ac640 1 -- 172.21.15.26:0/215492660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108419dae0 con 0x7f1084107270 2024-04-21T10:41:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.702+0000 7f10889ac640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1084107270 0x7f1084198c70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f107400d770 tx=0x7f107400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.702+0000 7f107a7fc640 1 -- 172.21.15.26:0/215492660 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074017070 con 0x7f1084107270 2024-04-21T10:41:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.702+0000 7f107a7fc640 1 -- 172.21.15.26:0/215492660 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1074009c30 con 0x7f1084107270 2024-04-21T10:41:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.702+0000 7f107a7fc640 1 -- 172.21.15.26:0/215492660 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107400f620 con 0x7f1084107270 2024-04-21T10:41:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.702+0000 7f108b411640 1 -- 172.21.15.26:0/215492660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108419ddd0 con 0x7f1084107270 2024-04-21T10:41:43.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.702+0000 7f108b411640 1 -- 172.21.15.26:0/215492660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10841a5610 con 0x7f1084107270 2024-04-21T10:41:43.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.703+0000 7f108b411640 1 -- 172.21.15.26:0/215492660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10840691c0 con 0x7f1084107270 2024-04-21T10:41:43.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.704+0000 7f107a7fc640 1 -- 172.21.15.26:0/215492660 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f107401a070 con 0x7f1084107270 2024-04-21T10:41:43.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.705+0000 7f107a7fc640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f10700a3da0 0x7f10700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T10:41:43.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.705+0000 7f10891ad640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f10700a3da0 0x7f10700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T10:41:43.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.705+0000 7f107a7fc640 1 -- 172.21.15.26:0/215492660 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8231+0+0 (secure 0 0 0) 0x7f10740d01d0 con 0x7f1084107270 2024-04-21T10:41:43.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.705+0000 7f10891ad640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f10700a3da0 0x7f10700a6260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1080036040 tx=0x7f1080000bc0 comp rx=0 tx=0).ready entity=mgr.24869 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T10:41:43.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.709+0000 7f107a7fc640 1 -- 172.21.15.26:0/215492660 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1074009de0 con 0x7f1084107270 2024-04-21T10:41:43.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:43.897+0000 7f108b411640 1 -- 172.21.15.26:0/215492660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f108406b1f0 con 0x7f1084107270 2024-04-21T10:41:43.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:43 smithi026 ceph-mon[27965]: pgmap v336: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-21T10:41:43.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:43 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3686316572' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T10:41:43.947 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:43 smithi169 ceph-mon[31469]: pgmap v336: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-21T10:41:43.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:43 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3686316572' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T10:41:44.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:43 smithi190 ceph-mon[31423]: pgmap v336: 97 pgs: 97 active+clean; 65 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-21T10:41:44.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:43 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3686316572' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T10:41:44.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.581+0000 7f107a7fc640 1 -- 172.21.15.26:0/215492660 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7f107409c990 con 0x7f1084107270 2024-04-21T10:41:44.581 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-04-21T10:41:44.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.584+0000 7f108b411640 1 -- 172.21.15.26:0/215492660 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f10700a3da0 msgr2=0x7f10700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:44.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.585+0000 7f108b411640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f10700a3da0 0x7f10700a6260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1080036040 tx=0x7f1080000bc0 comp rx=0 tx=0).stop 2024-04-21T10:41:44.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.585+0000 7f108b411640 1 -- 172.21.15.26:0/215492660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1084107270 msgr2=0x7f1084198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T10:41:44.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.585+0000 7f108b411640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1084107270 0x7f1084198c70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f107400d770 tx=0x7f107400dc30 comp rx=0 tx=0).stop 2024-04-21T10:41:44.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.585+0000 7f108b411640 1 -- 172.21.15.26:0/215492660 shutdown_connections 2024-04-21T10:41:44.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.585+0000 7f108b411640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.26:6800/4245046040,v1:172.21.15.26:6801/4245046040] conn(0x7f10700a3da0 0x7f10700a6260 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:44.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.585+0000 7f108b411640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f108419cff0 0x7f108419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:44.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.585+0000 7f108b411640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1084107270 0x7f1084198c70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:44.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.585+0000 7f108b411640 1 --2- 172.21.15.26:0/215492660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108406a840 0x7f1084198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T10:41:44.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.585+0000 7f108b411640 1 -- 172.21.15.26:0/215492660 >> 172.21.15.26:0/215492660 conn(0x7f1084077eb0 msgr2=0x7f1084108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T10:41:44.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.586+0000 7f108b411640 1 -- 172.21.15.26:0/215492660 shutdown_connections 2024-04-21T10:41:44.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-21T10:41:44.586+0000 7f108b411640 1 -- 172.21.15.26:0/215492660 wait complete. 2024-04-21T10:41:44.604 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T10:41:44.630 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T10:41:44.660 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T10:41:44.690 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-21T10:41:44.690 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T10:41:44.707 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T10:41:44.722 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T10:41:44.746 INFO:tasks.cephadm:Stopping all daemons... 2024-04-21T10:41:44.746 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-21T10:41:44.746 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.a 2024-04-21T10:41:44.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:44 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/3686316572' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T10:41:44.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:44 smithi026 ceph-mon[27965]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T10:41:44.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:44 smithi026 ceph-mon[27965]: from='client.? 172.21.15.26:0/215492660' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T10:41:44.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:44 smithi026 systemd[1]: Stopping Ceph mon.a for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:41:44.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:44 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/3686316572' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T10:41:44.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:44 smithi169 ceph-mon[31469]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T10:41:44.948 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:44 smithi169 ceph-mon[31469]: from='client.? 172.21.15.26:0/215492660' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T10:41:45.026 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:44 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/3686316572' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T10:41:45.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:44 smithi190 ceph-mon[31423]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T10:41:45.027 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:44 smithi190 ceph-mon[31423]: from='client.? 172.21.15.26:0/215492660' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T10:41:45.434 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:45 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a[27961]: 2024-04-21T10:41:45.074+0000 7f4ea1c75700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:41:45.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:45 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a[27961]: 2024-04-21T10:41:45.074+0000 7f4ea1c75700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-21T10:41:45.435 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:45 smithi026 podman[90859]: 2024-04-21 10:41:45.319931109 +0000 UTC m=+0.373186934 container died 1526264d8981cbd07a9d45e7c0d71bc634e1809f178ac78a012ae3044bff604f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-21T10:41:45.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:45 smithi026 podman[90859]: 2024-04-21 10:41:45.708158048 +0000 UTC m=+0.761413855 container cleanup 1526264d8981cbd07a9d45e7c0d71bc634e1809f178ac78a012ae3044bff604f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-21T10:41:45.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Apr 21 10:41:45 smithi026 bash[90859]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-a 2024-04-21T10:41:46.085 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.a.service' 2024-04-21T10:41:46.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:41:46.134 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-21T10:41:46.135 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-21T10:41:46.135 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.b 2024-04-21T10:41:46.433 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:46 smithi169 systemd[1]: Stopping Ceph mon.b for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:41:46.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:46 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-b[31465]: 2024-04-21T10:41:46.434+0000 7f46882cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:41:46.698 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:46 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-b[31465]: 2024-04-21T10:41:46.434+0000 7f46882cd700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-21T10:41:46.699 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:46 smithi169 podman[67536]: 2024-04-21 10:41:46.45546837 +0000 UTC m=+0.133429120 container died 4f5a02b35bd3c649960d8ea0e14a814e9571913c4a84f721350dac8cf0d551b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-21T10:41:47.084 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:46 smithi169 podman[67536]: 2024-04-21 10:41:46.867222383 +0000 UTC m=+0.545183136 container cleanup 4f5a02b35bd3c649960d8ea0e14a814e9571913c4a84f721350dac8cf0d551b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-21T10:41:47.084 INFO:journalctl@ceph.mon.b.smithi169.stdout:Apr 21 10:41:46 smithi169 bash[67536]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-b 2024-04-21T10:41:47.218 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.b.service' 2024-04-21T10:41:47.290 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:41:47.291 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-21T10:41:47.291 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-21T10:41:47.291 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.c 2024-04-21T10:41:47.588 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:47 smithi190 systemd[1]: Stopping Ceph mon.c for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:41:47.982 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:47 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-c[31397]: 2024-04-21T10:41:47.588+0000 7fdd16e0e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:41:47.982 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:47 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-c[31397]: 2024-04-21T10:41:47.588+0000 7fdd16e0e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-21T10:41:47.982 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:47 smithi190 podman[68266]: 2024-04-21 10:41:47.803721034 +0000 UTC m=+0.323068988 container died 5f7e1da45384ff7fdf5aba43c50caaf7a28ea11189e577a554aa9180001c3a96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T10:41:48.237 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:48 smithi190 podman[68266]: 2024-04-21 10:41:48.189164699 +0000 UTC m=+0.708512612 container cleanup 5f7e1da45384ff7fdf5aba43c50caaf7a28ea11189e577a554aa9180001c3a96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-21T10:41:48.237 INFO:journalctl@ceph.mon.c.smithi190.stdout:Apr 21 10:41:48 smithi190 bash[68266]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mon-c 2024-04-21T10:41:48.635 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mon.c.service' 2024-04-21T10:41:48.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:41:48.683 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-21T10:41:48.684 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-21T10:41:48.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.x 2024-04-21T10:41:49.037 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:41:48 smithi026 systemd[1]: Stopping Ceph mgr.x for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:41:49.370 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:41:49 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:41:49.036+0000 7f5526f01700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:41:49.370 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:41:49 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T10:41:49.037+0000 7f5526f01700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-21T10:41:49.370 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:41:49 smithi026 podman[90982]: 2024-04-21 10:41:49.06927427 +0000 UTC m=+0.183116680 container died ee7c00245b822c7bb7d424707caaddb7aede572f808ae9290a8cb11e3370bad0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T10:41:49.684 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:41:49 smithi026 podman[90982]: 2024-04-21 10:41:49.446398647 +0000 UTC m=+0.560241059 container cleanup ee7c00245b822c7bb7d424707caaddb7aede572f808ae9290a8cb11e3370bad0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-21T10:41:49.685 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Apr 21 10:41:49 smithi026 bash[90982]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-x 2024-04-21T10:41:49.879 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.x.service' 2024-04-21T10:41:49.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:41:49.915 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-21T10:41:49.915 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-21T10:41:49.915 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.y 2024-04-21T10:41:50.198 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:41:49 smithi169 systemd[1]: Stopping Ceph mgr.y for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:41:50.546 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:41:50 smithi169 podman[67668]: 2024-04-21 10:41:50.259135835 +0000 UTC m=+0.142671489 container died d677f561df8a4493485e27feaafa89e9911153540d9f963954afea6794dc0bef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:41:50.897 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:41:50 smithi169 podman[67668]: 2024-04-21 10:41:50.613737952 +0000 UTC m=+0.497273612 container cleanup d677f561df8a4493485e27feaafa89e9911153540d9f963954afea6794dc0bef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-21T10:41:50.897 INFO:journalctl@ceph.mgr.y.smithi169.stdout:Apr 21 10:41:50 smithi169 bash[67668]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-y 2024-04-21T10:41:51.040 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.y.service' 2024-04-21T10:41:51.074 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:41:51.074 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-21T10:41:51.074 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-21T10:41:51.075 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.z 2024-04-21T10:41:51.438 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:41:51 smithi190 systemd[1]: Stopping Ceph mgr.z for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:41:51.718 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:41:51 smithi190 podman[68398]: 2024-04-21 10:41:51.439671787 +0000 UTC m=+0.166440136 container died 189f62033133a8464ba88d15fcf061c150b7f15b1a070ca1d893906627cbe52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-21T10:41:52.026 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:41:51 smithi190 podman[68398]: 2024-04-21 10:41:51.818601146 +0000 UTC m=+0.545369495 container cleanup 189f62033133a8464ba88d15fcf061c150b7f15b1a070ca1d893906627cbe52f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-21T10:41:52.026 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Apr 21 10:41:51 smithi190 bash[68398]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mgr-z 2024-04-21T10:41:52.237 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mgr.z.service' 2024-04-21T10:41:52.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:41:52.270 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-21T10:41:52.270 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-21T10:41:52.271 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.0 2024-04-21T10:41:52.558 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:52 smithi026 systemd[1]: Stopping Ceph osd.0 for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:41:52.934 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:52 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0[39885]: 2024-04-21T10:41:52.558+0000 7fd674edd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:41:52.935 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:52 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0[39885]: 2024-04-21T10:41:52.558+0000 7fd674edd700 -1 osd.0 103 *** Got signal Terminated *** 2024-04-21T10:41:52.935 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:52 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0[39885]: 2024-04-21T10:41:52.558+0000 7fd674edd700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T10:41:58.134 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:57 smithi026 podman[91104]: 2024-04-21 10:41:57.878269445 +0000 UTC m=+5.411707124 container died 56c9fc28a68eb0580ef044df54d1d6138c33250a8d7bf0d60115f28b837bfc90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T10:41:58.434 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:58 smithi026 podman[91104]: 2024-04-21 10:41:58.35089211 +0000 UTC m=+5.884329786 container cleanup 56c9fc28a68eb0580ef044df54d1d6138c33250a8d7bf0d60115f28b837bfc90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:41:58.435 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:58 smithi026 bash[91104]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0 2024-04-21T10:41:58.907 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:58 smithi026 podman[91123]: 2024-04-21 10:41:58.610622271 +0000 UTC m=+0.733146247 container remove 56c9fc28a68eb0580ef044df54d1d6138c33250a8d7bf0d60115f28b837bfc90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-21T10:41:59.185 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:58 smithi026 podman[91209]: 2024-04-21 10:41:58.812791475 +0000 UTC m=+0.018669820 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:41:59.185 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:58 smithi026 podman[91209]: 2024-04-21 10:41:58.919683243 +0000 UTC m=+0.125561575 container create fa06fb9aff8a34cf11d09d44727a5d7eade01d4f32f1ccfd728d0c503f67b3ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-21T10:41:59.185 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:59 smithi026 podman[91209]: 2024-04-21 10:41:59.155351458 +0000 UTC m=+0.361229792 container init fa06fb9aff8a34cf11d09d44727a5d7eade01d4f32f1ccfd728d0c503f67b3ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-21T10:41:59.186 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:59 smithi026 podman[91209]: 2024-04-21 10:41:59.15863072 +0000 UTC m=+0.364509056 container start fa06fb9aff8a34cf11d09d44727a5d7eade01d4f32f1ccfd728d0c503f67b3ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T10:41:59.466 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:59 smithi026 podman[91209]: 2024-04-21 10:41:59.247292384 +0000 UTC m=+0.453170724 container attach fa06fb9aff8a34cf11d09d44727a5d7eade01d4f32f1ccfd728d0c503f67b3ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-21T10:41:59.765 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:41:59 smithi026 podman[91209]: 2024-04-21 10:41:59.46797537 +0000 UTC m=+0.673853710 container died fa06fb9aff8a34cf11d09d44727a5d7eade01d4f32f1ccfd728d0c503f67b3ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:42:00.017 INFO:journalctl@ceph.osd.0.smithi026.stdout:Apr 21 10:42:00 smithi026 podman[91239]: 2024-04-21 10:42:00.00949113 +0000 UTC m=+0.535476468 container remove fa06fb9aff8a34cf11d09d44727a5d7eade01d4f32f1ccfd728d0c503f67b3ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-21T10:42:00.023 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.0.service' 2024-04-21T10:42:00.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:42:00.063 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-21T10:42:00.063 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-21T10:42:00.063 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.1 2024-04-21T10:42:00.366 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:00 smithi169 systemd[1]: Stopping Ceph osd.1 for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:42:00.698 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:00 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1[36066]: 2024-04-21T10:42:00.367+0000 7fe88ab8e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:42:00.698 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:00 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1[36066]: 2024-04-21T10:42:00.367+0000 7fe88ab8e700 -1 osd.1 103 *** Got signal Terminated *** 2024-04-21T10:42:00.698 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:00 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1[36066]: 2024-04-21T10:42:00.367+0000 7fe88ab8e700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T10:42:06.197 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:05 smithi169 podman[67792]: 2024-04-21 10:42:05.920215766 +0000 UTC m=+5.655064140 container died 6b38493a1b26ed06c9234296ad1325419768955e50225a5083bfc93fde422ebf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-21T10:42:06.475 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:06 smithi169 podman[67792]: 2024-04-21 10:42:06.284037291 +0000 UTC m=+6.018885677 container cleanup 6b38493a1b26ed06c9234296ad1325419768955e50225a5083bfc93fde422ebf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-21T10:42:06.475 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:06 smithi169 bash[67792]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1 2024-04-21T10:42:06.905 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:06 smithi169 podman[67809]: 2024-04-21 10:42:06.476988683 +0000 UTC m=+0.560065255 container remove 6b38493a1b26ed06c9234296ad1325419768955e50225a5083bfc93fde422ebf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T10:42:07.187 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:06 smithi169 podman[67901]: 2024-04-21 10:42:06.810071618 +0000 UTC m=+0.018787830 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:42:07.188 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:06 smithi169 podman[67901]: 2024-04-21 10:42:06.936555362 +0000 UTC m=+0.145271573 container create 8ad8c7be1bec80aaceebe2ce6ddaa163f3d6dd9a05444ff41c3e785506840615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-21T10:42:07.448 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:07 smithi169 podman[67901]: 2024-04-21 10:42:07.189475898 +0000 UTC m=+0.398192110 container init 8ad8c7be1bec80aaceebe2ce6ddaa163f3d6dd9a05444ff41c3e785506840615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:42:07.448 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:07 smithi169 podman[67901]: 2024-04-21 10:42:07.192861267 +0000 UTC m=+0.401577480 container start 8ad8c7be1bec80aaceebe2ce6ddaa163f3d6dd9a05444ff41c3e785506840615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-21T10:42:07.448 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:07 smithi169 podman[67901]: 2024-04-21 10:42:07.306305352 +0000 UTC m=+0.515021577 container attach 8ad8c7be1bec80aaceebe2ce6ddaa163f3d6dd9a05444ff41c3e785506840615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-21T10:42:07.774 INFO:journalctl@ceph.osd.1.smithi169.stdout:Apr 21 10:42:07 smithi169 podman[67901]: 2024-04-21 10:42:07.495098788 +0000 UTC m=+0.703814999 container died 8ad8c7be1bec80aaceebe2ce6ddaa163f3d6dd9a05444ff41c3e785506840615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-21T10:42:08.052 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.1.service' 2024-04-21T10:42:08.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:42:08.090 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-21T10:42:08.091 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-21T10:42:08.091 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.2 2024-04-21T10:42:08.396 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:08 smithi190 systemd[1]: Stopping Ceph osd.2 for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:42:08.776 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:08 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2[36587]: 2024-04-21T10:42:08.395+0000 7f1db7d91700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:42:08.777 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:08 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2[36587]: 2024-04-21T10:42:08.395+0000 7f1db7d91700 -1 osd.2 103 *** Got signal Terminated *** 2024-04-21T10:42:08.777 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:08 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2[36587]: 2024-04-21T10:42:08.395+0000 7f1db7d91700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T10:42:14.121 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:13 smithi190 podman[68522]: 2024-04-21 10:42:13.849165961 +0000 UTC m=+5.565100500 container died 2ab2182e110dff699ed1cdfc55d78e12122b34d9f010db96a7812b96aa2fec50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-21T10:42:14.479 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:14 smithi190 podman[68522]: 2024-04-21 10:42:14.205247099 +0000 UTC m=+5.921181636 container cleanup 2ab2182e110dff699ed1cdfc55d78e12122b34d9f010db96a7812b96aa2fec50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-21T10:42:14.479 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:14 smithi190 bash[68522]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2 2024-04-21T10:42:14.776 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:14 smithi190 podman[68539]: 2024-04-21 10:42:14.48104096 +0000 UTC m=+0.631225025 container remove 2ab2182e110dff699ed1cdfc55d78e12122b34d9f010db96a7812b96aa2fec50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-21T10:42:15.132 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:14 smithi190 podman[68629]: 2024-04-21 10:42:14.815156794 +0000 UTC m=+0.111050706 container create e58e7e3ebe4a06acd7b9c823e442941c3ae587cd5c3c37cccbf5d80ccac2f617 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-21T10:42:15.133 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:14 smithi190 podman[68629]: 2024-04-21 10:42:14.722087605 +0000 UTC m=+0.017981546 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:42:15.133 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:15 smithi190 podman[68629]: 2024-04-21 10:42:15.050758674 +0000 UTC m=+0.346652590 container init e58e7e3ebe4a06acd7b9c823e442941c3ae587cd5c3c37cccbf5d80ccac2f617 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T10:42:15.133 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:15 smithi190 podman[68629]: 2024-04-21 10:42:15.05413914 +0000 UTC m=+0.350033060 container start e58e7e3ebe4a06acd7b9c823e442941c3ae587cd5c3c37cccbf5d80ccac2f617 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-21T10:42:15.133 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:15 smithi190 podman[68629]: 2024-04-21 10:42:15.13401159 +0000 UTC m=+0.429905512 container attach e58e7e3ebe4a06acd7b9c823e442941c3ae587cd5c3c37cccbf5d80ccac2f617 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-21T10:42:15.527 INFO:journalctl@ceph.osd.2.smithi190.stdout:Apr 21 10:42:15 smithi190 podman[68629]: 2024-04-21 10:42:15.340849865 +0000 UTC m=+0.636743789 container died e58e7e3ebe4a06acd7b9c823e442941c3ae587cd5c3c37cccbf5d80ccac2f617 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T10:42:15.955 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.2.service' 2024-04-21T10:42:15.993 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:42:15.994 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-21T10:42:15.994 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-21T10:42:15.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.3 2024-04-21T10:42:16.293 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:16 smithi026 systemd[1]: Stopping Ceph osd.3 for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:42:16.565 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:16 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3[46410]: 2024-04-21T10:42:16.293+0000 7fa875841700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:42:16.565 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:16 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3[46410]: 2024-04-21T10:42:16.293+0000 7fa875841700 -1 osd.3 103 *** Got signal Terminated *** 2024-04-21T10:42:16.565 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:16 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3[46410]: 2024-04-21T10:42:16.293+0000 7fa875841700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T10:42:19.685 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:19 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3[46410]: 2024-04-21T10:42:19.349+0000 7fa871e35700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.175393+0000 front 2024-04-21T10:41:56.175444+0000 (oldest deadline 2024-04-21T10:42:19.075533+0000) 2024-04-21T10:42:19.685 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:19 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:19.491+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:19.776 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:19 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:19.290+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:20.685 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:20 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3[46410]: 2024-04-21T10:42:20.398+0000 7fa871e35700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.175393+0000 front 2024-04-21T10:41:56.175444+0000 (oldest deadline 2024-04-21T10:42:19.075533+0000) 2024-04-21T10:42:20.685 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:20 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:20.538+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:20.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:20 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:20.469+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:20.776 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:20 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:20.301+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:21.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:20 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:20.744+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:21.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:21 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:21.274+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:21.902 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:21 smithi026 podman[91319]: 2024-04-21 10:42:21.618340892 +0000 UTC m=+5.424785179 container died dfe0383836b6116e6ff4683d2f92fbb4be8682b22b0b68ef77e153d2b6e3ef78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=) 2024-04-21T10:42:21.902 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:21 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:21.503+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:21.903 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:21 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:21.474+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:22.177 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:21 smithi026 podman[91319]: 2024-04-21 10:42:21.994333933 +0000 UTC m=+5.800778253 container cleanup dfe0383836b6116e6ff4683d2f92fbb4be8682b22b0b68ef77e153d2b6e3ef78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-21T10:42:22.177 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:21 smithi026 bash[91319]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3 2024-04-21T10:42:22.197 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:21 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:21.787+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:22.434 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:22 smithi026 podman[91349]: 2024-04-21 10:42:22.178841416 +0000 UTC m=+0.563008551 container remove dfe0383836b6116e6ff4683d2f92fbb4be8682b22b0b68ef77e153d2b6e3ef78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-21T10:42:22.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:22 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:22.252+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:22.775 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:22 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:22.502+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:22.776 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:22 smithi026 podman[91436]: 2024-04-21 10:42:22.403798655 +0000 UTC m=+0.019150995 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:42:22.776 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:22 smithi026 podman[91436]: 2024-04-21 10:42:22.529915452 +0000 UTC m=+0.145267780 container create 157ba418c9fed4031c3285d460592f4d2b5adab4c122672f11bb0d8323327622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-21T10:42:22.776 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:22 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:22.517+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:23.026 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:22 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:22.744+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:23.092 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:22 smithi026 podman[91436]: 2024-04-21 10:42:22.777401993 +0000 UTC m=+0.392754323 container init 157ba418c9fed4031c3285d460592f4d2b5adab4c122672f11bb0d8323327622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-21T10:42:23.093 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:22 smithi026 podman[91436]: 2024-04-21 10:42:22.780883493 +0000 UTC m=+0.396235825 container start 157ba418c9fed4031c3285d460592f4d2b5adab4c122672f11bb0d8323327622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:42:23.093 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:22 smithi026 podman[91436]: 2024-04-21 10:42:22.865663755 +0000 UTC m=+0.481016100 container attach 157ba418c9fed4031c3285d460592f4d2b5adab4c122672f11bb0d8323327622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-21T10:42:23.197 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:22 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:22.746+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:23.392 INFO:journalctl@ceph.osd.3.smithi026.stdout:Apr 21 10:42:23 smithi026 podman[91436]: 2024-04-21 10:42:23.094478049 +0000 UTC m=+0.709830368 container died 157ba418c9fed4031c3285d460592f4d2b5adab4c122672f11bb0d8323327622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-21T10:42:23.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:23 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:23.267+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:23.642 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:23 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:23.557+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:23.643 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:23 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:23.461+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:23.664 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.3.service' 2024-04-21T10:42:23.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:42:23.708 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-21T10:42:23.708 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-21T10:42:23.708 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.4 2024-04-21T10:42:24.004 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:23 smithi169 systemd[1]: Stopping Ceph osd.4 for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:42:24.004 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:23 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:23.733+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:24.026 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:23 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:23.706+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:24.447 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:24 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4[42562]: 2024-04-21T10:42:24.003+0000 7fbbb3183700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:42:24.448 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:24 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4[42562]: 2024-04-21T10:42:24.003+0000 7fbbb3183700 -1 osd.4 103 *** Got signal Terminated *** 2024-04-21T10:42:24.448 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:24 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4[42562]: 2024-04-21T10:42:24.003+0000 7fbbb3183700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T10:42:24.744 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:24 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:24.311+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:24.934 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:24 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:24.595+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:24.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:24 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:24.458+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:25.026 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:24 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:24.744+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:25.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:24 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:24.749+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:25.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:25 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:25.272+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:25.934 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:25 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:25.585+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:25.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:25 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:25.463+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:26.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:25 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:25.724+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:26.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:25 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:25.781+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:26.685 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:26 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:26.635+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:26.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:26 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:26.428+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:26.697 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:26 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4[42562]: 2024-04-21T10:42:26.427+0000 7fbbab76f700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.834662+0000 front 2024-04-21T10:42:02.834693+0000 (oldest deadline 2024-04-21T10:42:26.334903+0000) 2024-04-21T10:42:26.776 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:26 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:26.308+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:27.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:26 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:26.745+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:27.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:26 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:26.745+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:27.267 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:26 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:26.796+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:27.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:27 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:27.267+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:27.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:27 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:27.477+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:27.697 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:27 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4[42562]: 2024-04-21T10:42:27.392+0000 7fbbab76f700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.834662+0000 front 2024-04-21T10:42:02.834693+0000 (oldest deadline 2024-04-21T10:42:26.334903+0000) 2024-04-21T10:42:27.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:27 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:27.657+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:27.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:27 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:27.449+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:28.026 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:27 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:27.806+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:28.027 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:27 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:27.806+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.850095+0000 front 2024-04-21T10:42:02.850115+0000 (oldest deadline 2024-04-21T10:42:26.950187+0000) 2024-04-21T10:42:28.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:27 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:27.725+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:28.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:27 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:27.725+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:28.675 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:28 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4[42562]: 2024-04-21T10:42:28.401+0000 7fbbab76f700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.834662+0000 front 2024-04-21T10:42:02.834693+0000 (oldest deadline 2024-04-21T10:42:26.334903+0000) 2024-04-21T10:42:28.675 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:28 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:28.647+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:28.776 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:28 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:28.295+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:28.777 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:28 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:28.513+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:28.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:28 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:28.649+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:28.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:28 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:28.447+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:28.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:28 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:28.447+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:28.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:28 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:28.675+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:28.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:28 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:28.675+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:29.270 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:28 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:28.824+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:29.270 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:28 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:28.824+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.850095+0000 front 2024-04-21T10:42:02.850115+0000 (oldest deadline 2024-04-21T10:42:26.950187+0000) 2024-04-21T10:42:29.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:29 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:29.271+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:29.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:29 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:29.271+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:29.813 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:29 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:29.688+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:29.814 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:29 smithi169 podman[68019]: 2024-04-21 10:42:29.503632225 +0000 UTC m=+5.598012180 container died fe92112709ccd6e90ea247cbae675a51aebb2e68ab2651e9a0bf4495a2ab2057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-21T10:42:29.814 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:29 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:29.626+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:29.814 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:29 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:29.626+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:29.863 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:29 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:29.534+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:29.934 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:29 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:29.617+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:29.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:29 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:29.490+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:29.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:29 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:29.490+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:30.130 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:29 smithi169 podman[68019]: 2024-04-21 10:42:29.913715997 +0000 UTC m=+6.008095958 container cleanup fe92112709ccd6e90ea247cbae675a51aebb2e68ab2651e9a0bf4495a2ab2057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T10:42:30.130 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:29 smithi169 bash[68019]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4 2024-04-21T10:42:30.245 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:29 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:29.864+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:30.246 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:29 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:29.864+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.850095+0000 front 2024-04-21T10:42:02.850115+0000 (oldest deadline 2024-04-21T10:42:26.950187+0000) 2024-04-21T10:42:30.447 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:30 smithi169 podman[68034]: 2024-04-21 10:42:30.131664875 +0000 UTC m=+0.625987904 container remove fe92112709ccd6e90ea247cbae675a51aebb2e68ab2651e9a0bf4495a2ab2057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4, RELEASE=wip-lusov-quiescer-a630765, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T10:42:30.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:30 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:30.247+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:30.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:30 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:30.247+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:30.835 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:30 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:30.670+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:30.835 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:30 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:30.664+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:30.835 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:30 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:30.664+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:30.836 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:30 smithi169 podman[68127]: 2024-04-21 10:42:30.437301184 +0000 UTC m=+0.018150032 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:42:30.836 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:30 smithi169 podman[68127]: 2024-04-21 10:42:30.557807793 +0000 UTC m=+0.138656627 container create 21edfe0279da335fe6960b5fb70a406d446df667f9830aab2b4565f5d2554a56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:42:30.836 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:30 smithi169 podman[68127]: 2024-04-21 10:42:30.761888316 +0000 UTC m=+0.342737157 container init 21edfe0279da335fe6960b5fb70a406d446df667f9830aab2b4565f5d2554a56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-21T10:42:30.836 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:30 smithi169 podman[68127]: 2024-04-21 10:42:30.765392254 +0000 UTC m=+0.346241090 container start 21edfe0279da335fe6960b5fb70a406d446df667f9830aab2b4565f5d2554a56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T10:42:30.836 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:30 smithi169 podman[68127]: 2024-04-21 10:42:30.836171922 +0000 UTC m=+0.417020770 container attach 21edfe0279da335fe6960b5fb70a406d446df667f9830aab2b4565f5d2554a56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-21T10:42:30.875 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:30 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:30.558+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:30.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:30 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:30.575+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:30.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:30 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:30.575+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:30.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:30 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:30.502+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:30.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:30 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:30.502+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:31.197 INFO:journalctl@ceph.osd.4.smithi169.stdout:Apr 21 10:42:31 smithi169 podman[68127]: 2024-04-21 10:42:31.05594432 +0000 UTC m=+0.636793160 container died 21edfe0279da335fe6960b5fb70a406d446df667f9830aab2b4565f5d2554a56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-21T10:42:31.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:30 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:30.876+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:31.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:30 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:30.876+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.850095+0000 front 2024-04-21T10:42:02.850115+0000 (oldest deadline 2024-04-21T10:42:26.950187+0000) 2024-04-21T10:42:31.561 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:31 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:31.284+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:31.562 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:31 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:31.284+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:31.659 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.4.service' 2024-04-21T10:42:31.699 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:42:31.699 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-21T10:42:31.699 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-21T10:42:31.699 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.5 2024-04-21T10:42:31.896 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:31 smithi190 systemd[1]: Stopping Ceph osd.5 for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:42:31.896 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:31 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:31.563+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:31.934 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:31 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:31.534+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:31.934 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:31 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:31.534+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:31.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:31 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:31.518+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:31.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:31 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:31.518+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:31.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:31 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:31.621+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:31.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:31 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:31.706+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:31.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:31 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:31.706+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:32.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:31 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:31.897+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:32.277 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:31 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:31.897+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.850095+0000 front 2024-04-21T10:42:02.850115+0000 (oldest deadline 2024-04-21T10:42:26.950187+0000) 2024-04-21T10:42:32.277 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:32 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:32.006+0000 7fe903a69700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:42:32.277 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:32 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:32.006+0000 7fe903a69700 -1 osd.5 103 *** Got signal Terminated *** 2024-04-21T10:42:32.277 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:32 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:32.006+0000 7fe903a69700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T10:42:32.559 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:32 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:32.301+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:32.560 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:32 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:32.301+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:32.934 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:32 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:32.492+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:32.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:32 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:32.492+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:32.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:32 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:32.509+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:32.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:32 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:32.509+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:32.941 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:32 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:32.560+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:32.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:32 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:32.600+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:32.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:32 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:32.705+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:32.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:32 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:32.705+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:33.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:32 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:32.942+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:33.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:32 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:32.942+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.850095+0000 front 2024-04-21T10:42:02.850115+0000 (oldest deadline 2024-04-21T10:42:26.950187+0000) 2024-04-21T10:42:33.563 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:33 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:33.279+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:33.563 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:33 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:33.279+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:33.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:33 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:33.512+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:33.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:33 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:33.512+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:33.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:33 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:33.496+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:33.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:33 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:33.496+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:33.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:33 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:33.583+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:33.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:33 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:33.684+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:33.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:33 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:33.684+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:33.968 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:33 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:33.564+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:34.244 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:33 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:33.969+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:34.244 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:33 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:33.969+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.850095+0000 front 2024-04-21T10:42:02.850115+0000 (oldest deadline 2024-04-21T10:42:26.950187+0000) 2024-04-21T10:42:34.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:34 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:34.245+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:34.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:34 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:34.245+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:34.934 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:34 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:34.530+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:34.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:34 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:34.530+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:34.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:34 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:34.554+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:34.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:34 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:34.554+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:34.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:34 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:34.538+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:34.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:34 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:34.722+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:34.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:34 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:34.722+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:34.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:34 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:34.722+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:35.013 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:34 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:34.546+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:35.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:35 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:35.014+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:35.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:35 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:35.014+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.850095+0000 front 2024-04-21T10:42:02.850115+0000 (oldest deadline 2024-04-21T10:42:26.950187+0000) 2024-04-21T10:42:35.646 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:35 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:35.509+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:35.647 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:35 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:35.509+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:35.647 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:35 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:35.292+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:35.647 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:35 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:35.292+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:35.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:35 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:35.552+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:35.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:35 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:35.552+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:35.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:35 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:35.511+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:35.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:35 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:35.511+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:35.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:35 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:35.511+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:12.540590+0000 front 2024-04-21T10:42:12.540554+0000 (oldest deadline 2024-04-21T10:42:35.440587+0000) 2024-04-21T10:42:35.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:35 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:35.573+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:35.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:35 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:35.767+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:35.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:35 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:35.767+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:35.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:35 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:35.767+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:36.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:36 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:36.011+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:36.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:36 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:36.011+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.850095+0000 front 2024-04-21T10:42:02.850115+0000 (oldest deadline 2024-04-21T10:42:26.950187+0000) 2024-04-21T10:42:36.776 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:36 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:36.537+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:36.777 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:36 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:36.537+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:36.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:36 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:36.289+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:36.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:36 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:36.289+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:36.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:36 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:36.289+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:36.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:36 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:36.537+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:36.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:36 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:36.537+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:36.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:36 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:36.553+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:36.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:36 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:36.553+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:36.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:36 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:36.553+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:12.540590+0000 front 2024-04-21T10:42:12.540554+0000 (oldest deadline 2024-04-21T10:42:35.440587+0000) 2024-04-21T10:42:36.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:36 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:36.543+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:36.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:36 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:36.543+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:36.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:36 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:36.772+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:36.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:36 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:36.772+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:36.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:36 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:36.772+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:37.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:36 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:36.967+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:57.049370+0000 front 2024-04-21T10:41:57.049236+0000 (oldest deadline 2024-04-21T10:42:22.349196+0000) 2024-04-21T10:42:37.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:36 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:36.967+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:02.850095+0000 front 2024-04-21T10:42:02.850115+0000 (oldest deadline 2024-04-21T10:42:26.950187+0000) 2024-04-21T10:42:37.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:36 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5[43076]: 2024-04-21T10:42:36.967+0000 7fe8fc055700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:12.751605+0000 front 2024-04-21T10:42:12.751650+0000 (oldest deadline 2024-04-21T10:42:36.251815+0000) 2024-04-21T10:42:37.539 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:37 smithi190 podman[68750]: 2024-04-21 10:42:37.492723019 +0000 UTC m=+5.597492552 container died a23a31fdfbccb93642d3b65317f0141df5348fa19591f0b8a162883ad4d90b62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-21T10:42:37.539 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:37 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:37.279+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:37.539 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:37 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:37.279+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:37.539 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:37 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:37.279+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:37.854 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:37 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:37.538+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:37.854 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:37 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:37.538+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:37.855 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:37 smithi190 podman[68750]: 2024-04-21 10:42:37.85599203 +0000 UTC m=+5.960761571 container cleanup a23a31fdfbccb93642d3b65317f0141df5348fa19591f0b8a162883ad4d90b62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-21T10:42:37.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:37 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:37.498+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:37.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:37 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:37.498+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:37.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:37 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:37.516+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:37.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:37 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:37.516+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:37.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:37 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:37.516+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:12.540590+0000 front 2024-04-21T10:42:12.540554+0000 (oldest deadline 2024-04-21T10:42:35.440587+0000) 2024-04-21T10:42:37.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:37 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:37.521+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:37.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:37 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:37.521+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:37.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:37 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:37.761+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:37.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:37 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:37.761+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:37.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:37 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:37.761+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:38.155 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:37 smithi190 bash[68750]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5 2024-04-21T10:42:38.470 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:38 smithi190 podman[68767]: 2024-04-21 10:42:38.157099942 +0000 UTC m=+0.659666141 container remove a23a31fdfbccb93642d3b65317f0141df5348fa19591f0b8a162883ad4d90b62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:42:38.470 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:38 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:38.285+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:38.470 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:38 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:38.285+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:38.470 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:38 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:38.285+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:38.777 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:38 smithi190 podman[68856]: 2024-04-21 10:42:38.374796259 +0000 UTC m=+0.018104723 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:42:38.777 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:38 smithi190 podman[68856]: 2024-04-21 10:42:38.482936271 +0000 UTC m=+0.126244723 container create 7a85a460b9c4f65dfb638770fb13b1ed06e153db18b10f9cc9b9b7fdf591b9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-21T10:42:38.777 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:38 smithi190 podman[68856]: 2024-04-21 10:42:38.702088317 +0000 UTC m=+0.345396766 container init 7a85a460b9c4f65dfb638770fb13b1ed06e153db18b10f9cc9b9b7fdf591b9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-21T10:42:38.777 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:38 smithi190 podman[68856]: 2024-04-21 10:42:38.705176595 +0000 UTC m=+0.348485050 container start 7a85a460b9c4f65dfb638770fb13b1ed06e153db18b10f9cc9b9b7fdf591b9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-21T10:42:38.777 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:38 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:38.510+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:38.778 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:38 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:38.510+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:38.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:38 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:38.528+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:38.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:38 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:38.528+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:38.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:38 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:38.528+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:38.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:38 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:38.549+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:38.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:38 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:38.549+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:38.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:38 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:38.549+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:12.540590+0000 front 2024-04-21T10:42:12.540554+0000 (oldest deadline 2024-04-21T10:42:35.440587+0000) 2024-04-21T10:42:38.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:38 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:38.535+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:38.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:38 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:38.535+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:38.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:38 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:38.753+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:38.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:38 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:38.753+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:38.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:38 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:38.753+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:39.276 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:38 smithi190 podman[68856]: 2024-04-21 10:42:38.785178947 +0000 UTC m=+0.428487404 container attach 7a85a460b9c4f65dfb638770fb13b1ed06e153db18b10f9cc9b9b7fdf591b9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-21T10:42:39.277 INFO:journalctl@ceph.osd.5.smithi190.stdout:Apr 21 10:42:39 smithi190 podman[68856]: 2024-04-21 10:42:39.003679636 +0000 UTC m=+0.646988087 container died 7a85a460b9c4f65dfb638770fb13b1ed06e153db18b10f9cc9b9b7fdf591b9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-21T10:42:39.703 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:39 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:39.499+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:39.704 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:39 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:39.499+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:39.704 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:39 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:39.292+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:39.704 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:39 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:39.292+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:39.704 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:39 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:39.292+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:39.733 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.5.service' 2024-04-21T10:42:39.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:42:39.770 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-21T10:42:39.770 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-21T10:42:39.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.6 2024-04-21T10:42:39.796 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:39 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:39.529+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:39.817 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:39 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:39.529+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:39.817 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:39 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:39.529+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:12.540590+0000 front 2024-04-21T10:42:12.540554+0000 (oldest deadline 2024-04-21T10:42:35.440587+0000) 2024-04-21T10:42:39.817 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:39 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:39.494+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:39.817 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:39 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:39.494+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:39.817 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:39 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:39.494+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:39.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:39 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:39.540+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:39.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:39 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:39.540+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:39.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:39 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:39.781+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:39.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:39 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:39.781+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:39.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:39 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:39.781+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:40.070 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:39 smithi026 systemd[1]: Stopping Ceph osd.6 for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:42:40.434 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:40.071+0000 7fd09645d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:42:40.435 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:40.071+0000 7fd09645d700 -1 osd.6 103 *** Got signal Terminated *** 2024-04-21T10:42:40.435 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:40.071+0000 7fd09645d700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T10:42:40.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:40 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:40.489+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:40.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:40 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:40.489+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:40.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:40 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:40.268+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:40.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:40 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:40.268+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:40.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:40 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:40.268+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:40.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:40.472+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:40.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:40.472+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:40.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:40.472+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:40.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:40.510+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:40.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:40.510+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:40.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:40.510+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:12.540590+0000 front 2024-04-21T10:42:12.540554+0000 (oldest deadline 2024-04-21T10:42:35.440587+0000) 2024-04-21T10:42:40.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:40 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:40.580+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:40.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:40 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:40.580+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:40.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:40 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:40.828+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:40.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:40 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:40.828+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:40.948 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:40 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:40.828+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:41.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:41.495+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:41.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:41.495+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:41.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:41.264+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:41.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:41.264+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:41.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:41 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:41.264+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:41.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:41 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:41.430+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:41.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:41 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:41.430+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:41.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:41 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:41.430+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:41.685 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:41 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:41.465+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:41.685 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:41 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:41.465+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:41.686 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:41 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:41.465+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:12.540590+0000 front 2024-04-21T10:42:12.540554+0000 (oldest deadline 2024-04-21T10:42:35.440587+0000) 2024-04-21T10:42:41.877 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:41.543+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:41.877 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:41.543+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:42.197 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:41.878+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:42.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:41.878+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:42.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:41 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:41.878+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:42.776 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:42 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:42.502+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:42.777 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:42 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:42.502+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:42.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:42 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:42.302+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:42.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:42 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:42.302+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:42.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:42 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:42.302+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:42.915 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:42 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:42.584+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:42.915 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:42 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:42.584+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:42.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:42 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:42.443+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:42.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:42 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:42.443+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:42.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:42 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:42.443+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:42.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:42 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:42.488+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:42.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:42 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:42.488+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:42.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:42 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:42.488+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:12.540590+0000 front 2024-04-21T10:42:12.540554+0000 (oldest deadline 2024-04-21T10:42:35.440587+0000) 2024-04-21T10:42:43.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:42 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:42.916+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:43.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:42 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:42.916+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:43.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:42 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:42.916+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:43.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:42 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:42.916+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:43.776 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:43 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:43.497+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:43.777 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:43 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:43.497+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:43.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:43 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:43.336+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:43.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:43 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:43.336+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:43.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:43 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:43.336+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:43.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:43 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:43.522+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:43.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:43 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:43.522+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:43.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:43 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:43.522+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:12.540590+0000 front 2024-04-21T10:42:12.540554+0000 (oldest deadline 2024-04-21T10:42:35.440587+0000) 2024-04-21T10:42:43.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:43 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:43.461+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:43.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:43 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:43.461+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:43.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:43 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:43.461+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:43.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:43 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:43.461+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:43.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:43 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:43.566+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:43.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:43 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:43.566+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:43.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:43 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:43.566+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:20.334763+0000 front 2024-04-21T10:42:20.334712+0000 (oldest deadline 2024-04-21T10:42:42.634551+0000) 2024-04-21T10:42:44.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:43 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:43.951+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:44.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:43 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:43.951+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:44.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:43 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:43.951+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:44.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:43 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:43.951+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:44.776 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:44 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:44.504+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:44.777 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:44 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:44.504+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:44.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:44 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:44.288+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:44.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:44 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:44.288+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:44.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:44 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:44.288+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:44.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:44 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:44.288+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:44.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:44 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:44.513+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:56.337304+0000 front 2024-04-21T10:41:56.337345+0000 (oldest deadline 2024-04-21T10:42:18.637426+0000) 2024-04-21T10:42:44.935 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:44 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:44.513+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.538499+0000 front 2024-04-21T10:42:04.538429+0000 (oldest deadline 2024-04-21T10:42:29.838431+0000) 2024-04-21T10:42:44.936 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:44 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6[52979]: 2024-04-21T10:42:44.513+0000 7fd092a51700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:12.540590+0000 front 2024-04-21T10:42:12.540554+0000 (oldest deadline 2024-04-21T10:42:35.440587+0000) 2024-04-21T10:42:44.936 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:44 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:44.438+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:44.937 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:44 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:44.438+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:44.937 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:44 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:44.438+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:44.937 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:44 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:44.438+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:44.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:44 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:44.571+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:44.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:44 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:44.571+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:44.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:44 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:44.571+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:20.334763+0000 front 2024-04-21T10:42:20.334712+0000 (oldest deadline 2024-04-21T10:42:42.634551+0000) 2024-04-21T10:42:45.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:44 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:44.958+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:45.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:44 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:44.958+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:45.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:44 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:44.958+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:45.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:44 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:44.958+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:45.541 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:45 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:45.290+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:45.541 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:45 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:45.290+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:45.541 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:45 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:45.290+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:45.541 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:45 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:45.290+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:45.685 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:45 smithi026 podman[91543]: 2024-04-21 10:42:45.490984969 +0000 UTC m=+5.524859828 container died 4bedfe05938533ef448655d7706dad7a5c35ea52266e2e139249d57b497437e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:42:45.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:45 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:45.434+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:45.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:45 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:45.434+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:45.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:45 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:45.434+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:45.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:45 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:45.434+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:45.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:45 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:45.530+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:45.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:45 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:45.530+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:45.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:45 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:45.530+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:20.334763+0000 front 2024-04-21T10:42:20.334712+0000 (oldest deadline 2024-04-21T10:42:42.634551+0000) 2024-04-21T10:42:46.026 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:45 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:45.541+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:46.027 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:45 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:45.541+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:46.137 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:45 smithi026 podman[91543]: 2024-04-21 10:42:45.829244883 +0000 UTC m=+5.863119729 container cleanup 4bedfe05938533ef448655d7706dad7a5c35ea52266e2e139249d57b497437e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-21T10:42:46.137 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:45 smithi026 bash[91543]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6 2024-04-21T10:42:46.427 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:46 smithi026 podman[91561]: 2024-04-21 10:42:46.13879026 +0000 UTC m=+0.642486937 container remove 4bedfe05938533ef448655d7706dad7a5c35ea52266e2e139249d57b497437e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-21T10:42:46.447 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:45 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:45.992+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:46.447 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:45 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:45.992+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:46.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:45 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:45.992+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:46.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:45 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:45.992+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:46.514 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:46 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:46.263+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:46.514 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:46 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:46.263+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:46.515 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:46 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:46.263+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:46.515 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:46 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:46.263+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:46.684 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:46 smithi026 podman[91650]: 2024-04-21 10:42:46.589702085 +0000 UTC m=+0.096047017 container create e3a5300aaff8af69f498e8051b138a555a2325c818cecf550a990dcdc7077a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-21T10:42:46.685 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:46 smithi026 podman[91650]: 2024-04-21 10:42:46.511917657 +0000 UTC m=+0.018262547 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:42:46.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:46 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:46.427+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:46.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:46 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:46.427+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:46.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:46 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:46.427+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:46.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:46 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:46.427+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:46.776 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:46 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:46.515+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:46.776 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:46 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:46.515+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:46.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:46 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:46.504+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:46.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:46 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:46.504+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:46.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:46 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:46.504+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:20.334763+0000 front 2024-04-21T10:42:20.334712+0000 (oldest deadline 2024-04-21T10:42:42.634551+0000) 2024-04-21T10:42:47.113 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:46 smithi026 podman[91650]: 2024-04-21 10:42:46.801317564 +0000 UTC m=+0.307662437 container init e3a5300aaff8af69f498e8051b138a555a2325c818cecf550a990dcdc7077a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-21T10:42:47.113 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:46 smithi026 podman[91650]: 2024-04-21 10:42:46.804855963 +0000 UTC m=+0.311200840 container start e3a5300aaff8af69f498e8051b138a555a2325c818cecf550a990dcdc7077a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-21T10:42:47.113 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:46 smithi026 podman[91650]: 2024-04-21 10:42:46.883767578 +0000 UTC m=+0.390112463 container attach e3a5300aaff8af69f498e8051b138a555a2325c818cecf550a990dcdc7077a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-21T10:42:47.395 INFO:journalctl@ceph.osd.6.smithi026.stdout:Apr 21 10:42:47 smithi026 podman[91650]: 2024-04-21 10:42:47.114974658 +0000 UTC m=+0.621319530 container died e3a5300aaff8af69f498e8051b138a555a2325c818cecf550a990dcdc7077a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T10:42:47.447 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:47 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:47.022+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:47.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:47 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:47.022+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:47.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:47 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:47.022+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:47.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:47 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:47.022+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:47.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:47 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:47.470+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:47.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:47 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:47.470+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:47.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:47 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:47.470+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:47.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:47 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:47.276+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:47.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:47 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:47.276+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:47.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:47 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:47.276+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:47.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:47 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:47.276+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:47.652 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:47 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:47.395+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:47.652 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:47 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:47.395+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:47.653 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:47 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:47.395+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:47.653 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:47 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:47.395+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:47.671 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.6.service' 2024-04-21T10:42:47.750 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:42:47.750 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-21T10:42:47.751 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-21T10:42:47.751 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.7 2024-04-21T10:42:47.776 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:47 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:47.471+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:47.776 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:47 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:47.471+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:47.777 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:47 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:47.471+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:20.334763+0000 front 2024-04-21T10:42:20.334712+0000 (oldest deadline 2024-04-21T10:42:42.634551+0000) 2024-04-21T10:42:48.041 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:47 smithi169 systemd[1]: Stopping Ceph osd.7 for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:42:48.042 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:48.004+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:48.042 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:48.004+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:48.042 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:48.004+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:48.042 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:48.004+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:48.438 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:48.042+0000 7f24f4e70700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:42:48.438 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:48.042+0000 7f24f4e70700 -1 osd.7 103 *** Got signal Terminated *** 2024-04-21T10:42:48.439 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:48.042+0000 7f24f4e70700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T10:42:48.439 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:48.439+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:48.439 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:48.439+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:48.439 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:48.439+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:20.334763+0000 front 2024-04-21T10:42:20.334712+0000 (oldest deadline 2024-04-21T10:42:42.634551+0000) 2024-04-21T10:42:48.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:48 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:48.430+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:48.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:48 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:48.430+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:48.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:48 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:48.430+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:48.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:48 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:48.237+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:48.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:48 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:48.237+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:48.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:48 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:48.237+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:48.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:48 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:48.237+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:48.684 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:48 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:48.413+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:48.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:48 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:48.413+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:48.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:48 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:48.413+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:48.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:48 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:48.413+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:49.445 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:48.960+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:49.446 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:48.960+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:49.446 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:48.960+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:49.446 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:48 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:48.960+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:49.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:49 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:49.424+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:49.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:49 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:49.424+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:49.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:49 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:49.424+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:49.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:49 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:49.230+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:49.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:49 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:49.230+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:49.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:49 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:49.230+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:49.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:49 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:49.230+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:49.697 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:49 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:49.446+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:49.698 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:49 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:49.446+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:49.698 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:49 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:49.446+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:20.334763+0000 front 2024-04-21T10:42:20.334712+0000 (oldest deadline 2024-04-21T10:42:42.634551+0000) 2024-04-21T10:42:49.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:49 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:49.436+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:49.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:49 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:49.436+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:49.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:49 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:49.436+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:49.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:49 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:49.436+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:50.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:49 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:49.954+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:50.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:49 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:49.954+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:50.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:49 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:49.954+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:50.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:49 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:49.954+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:50.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:50 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:50.411+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:50.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:50 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:50.411+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:50.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:50 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:50.411+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:50.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:50 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:50.244+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:50.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:50 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:50.244+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:50.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:50 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:50.244+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:50.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:50 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:50.244+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:50.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:50 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:50.420+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:50.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:50 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:50.420+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:50.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:50 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:50.420+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:50.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:50 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:50.420+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:50.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:50.456+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:50.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:50.456+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:50.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:50.456+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:20.334763+0000 front 2024-04-21T10:42:20.334712+0000 (oldest deadline 2024-04-21T10:42:42.634551+0000) 2024-04-21T10:42:51.414 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:50.974+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:51.414 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:50.974+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:51.414 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:50.974+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:51.414 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:50.974+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:51.414 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:50 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:50.974+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:42:51.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:51 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:51.364+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:51.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:51 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:51.364+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:51.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:51 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:51.364+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:51.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:51 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:51.364+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.753770+0000 front 2024-04-21T10:42:28.753878+0000 (oldest deadline 2024-04-21T10:42:51.053718+0000) 2024-04-21T10:42:51.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:51 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:51.234+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:51.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:51 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:51.234+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:51.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:51 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:51.234+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:51.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:51 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:51.234+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:51.684 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:51 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:51.405+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:51.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:51 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:51.405+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:51.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:51 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:51.405+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:51.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:51 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:51.405+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:51.697 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:51 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:51.415+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:51.698 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:51 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:51.415+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:51.698 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:51 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:51.415+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:20.334763+0000 front 2024-04-21T10:42:20.334712+0000 (oldest deadline 2024-04-21T10:42:42.634551+0000) 2024-04-21T10:42:51.698 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:51 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:51.415+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:27.335547+0000 front 2024-04-21T10:42:27.335570+0000 (oldest deadline 2024-04-21T10:42:50.835711+0000) 2024-04-21T10:42:52.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:51 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:51.933+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:52.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:51 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:51.933+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:52.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:51 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:51.933+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:52.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:51 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:51.933+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:52.198 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:51 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:51.933+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:42:52.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:52 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:52.379+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:52.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:52 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:52.379+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:52.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:52 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:52.379+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:52.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:52 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:52.379+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.753770+0000 front 2024-04-21T10:42:28.753878+0000 (oldest deadline 2024-04-21T10:42:51.053718+0000) 2024-04-21T10:42:52.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:52 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:52.202+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:52.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:52 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:52.202+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:52.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:52 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:52.202+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:52.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:52 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:52.202+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:52.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:52 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:52.440+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:52.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:52 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:52.440+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:52.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:52 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:52.440+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:52.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:52 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:52.440+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:52.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:52 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:52.458+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.932051+0000 front 2024-04-21T10:42:03.932216+0000 (oldest deadline 2024-04-21T10:42:28.032258+0000) 2024-04-21T10:42:52.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:52 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:52.458+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.533394+0000 front 2024-04-21T10:42:11.533416+0000 (oldest deadline 2024-04-21T10:42:35.633508+0000) 2024-04-21T10:42:52.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:52 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:52.458+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:20.334763+0000 front 2024-04-21T10:42:20.334712+0000 (oldest deadline 2024-04-21T10:42:42.634551+0000) 2024-04-21T10:42:52.948 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:52 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7[49590]: 2024-04-21T10:42:52.458+0000 7f24f1464700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:27.335547+0000 front 2024-04-21T10:42:27.335570+0000 (oldest deadline 2024-04-21T10:42:50.835711+0000) 2024-04-21T10:42:53.429 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:52 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:52.968+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:53.429 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:52 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:52.968+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:53.430 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:52 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:52.968+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:53.430 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:52 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:52.969+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:53.430 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:52 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:52.969+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:42:53.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:53.390+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:53.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:53.390+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:53.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:53.390+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:53.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:53.390+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.753770+0000 front 2024-04-21T10:42:28.753878+0000 (oldest deadline 2024-04-21T10:42:51.053718+0000) 2024-04-21T10:42:53.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:53.222+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:53.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:53.222+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:53.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:53.222+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:53.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:53.222+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:53.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:53 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:53.222+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:42:53.684 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:53 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:53.425+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:53.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:53 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:53.425+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:53.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:53 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:53.425+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:53.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:53 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:53.425+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:53.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:53 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:53.425+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:42:53.697 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:53 smithi169 podman[68233]: 2024-04-21 10:42:53.429705158 +0000 UTC m=+5.489452857 container died 31b0e85c263653717eb7bd7e54e7766d4b700f344599f383dea1f10e1eaaad20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-21T10:42:54.017 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:53 smithi169 podman[68233]: 2024-04-21 10:42:53.803924257 +0000 UTC m=+5.863671956 container cleanup 31b0e85c263653717eb7bd7e54e7766d4b700f344599f383dea1f10e1eaaad20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-21T10:42:54.018 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:53 smithi169 bash[68233]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7 2024-04-21T10:42:54.329 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:54 smithi169 podman[68265]: 2024-04-21 10:42:54.046741273 +0000 UTC m=+0.611294543 container remove 31b0e85c263653717eb7bd7e54e7766d4b700f344599f383dea1f10e1eaaad20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, ceph=True) 2024-04-21T10:42:54.330 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:54.018+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:54.330 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:54.018+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:54.330 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:54.018+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:54.330 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:54.018+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:54.330 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:54.018+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:42:54.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:54.413+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:54.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:54.413+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:54.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:54.413+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:54.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:54.413+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.753770+0000 front 2024-04-21T10:42:28.753878+0000 (oldest deadline 2024-04-21T10:42:51.053718+0000) 2024-04-21T10:42:54.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:54.251+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:54.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:54.251+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:54.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:54.251+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:54.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:54.251+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:54.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:54 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:54.251+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:42:54.657 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:54 smithi169 podman[68353]: 2024-04-21 10:42:54.331046205 +0000 UTC m=+0.105105871 container create 613dd1ad6d0ca5de4e68044e71d85a66f14d331c16a625005bc1d35940ea506b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-21T10:42:54.657 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:54 smithi169 podman[68353]: 2024-04-21 10:42:54.244309097 +0000 UTC m=+0.018368768 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:42:54.657 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:54 smithi169 podman[68353]: 2024-04-21 10:42:54.575420693 +0000 UTC m=+0.349480354 container init 613dd1ad6d0ca5de4e68044e71d85a66f14d331c16a625005bc1d35940ea506b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-21T10:42:54.657 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:54 smithi169 podman[68353]: 2024-04-21 10:42:54.579758317 +0000 UTC m=+0.353817985 container start 613dd1ad6d0ca5de4e68044e71d85a66f14d331c16a625005bc1d35940ea506b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True) 2024-04-21T10:42:54.657 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:54 smithi169 podman[68353]: 2024-04-21 10:42:54.658476906 +0000 UTC m=+0.432536568 container attach 613dd1ad6d0ca5de4e68044e71d85a66f14d331c16a625005bc1d35940ea506b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7-deactivate, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-21T10:42:54.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:54 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:54.463+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:54.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:54 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:54.463+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:54.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:54 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:54.463+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:54.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:54 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:54.463+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:54.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:54 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:54.463+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:42:54.947 INFO:journalctl@ceph.osd.7.smithi169.stdout:Apr 21 10:42:54 smithi169 podman[68353]: 2024-04-21 10:42:54.877050456 +0000 UTC m=+0.651110120 container died 613dd1ad6d0ca5de4e68044e71d85a66f14d331c16a625005bc1d35940ea506b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-04-21T10:42:55.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:54.981+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:55.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:54.981+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:55.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:54.981+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:55.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:54.981+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:55.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:54.981+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:42:55.518 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.7.service' 2024-04-21T10:42:55.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:55.434+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:55.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:55.434+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:55.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:55.434+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:55.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:55.434+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.753770+0000 front 2024-04-21T10:42:28.753878+0000 (oldest deadline 2024-04-21T10:42:51.053718+0000) 2024-04-21T10:42:55.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:55.261+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:55.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:55.261+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:55.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:55.261+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:55.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:55.261+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:55.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:55.261+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:42:55.598 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:42:55.599 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-21T10:42:55.599 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-21T10:42:55.599 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.8 2024-04-21T10:42:55.898 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:55 smithi190 systemd[1]: Stopping Ceph osd.8 for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:42:55.898 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:55.898+0000 7fddc1691700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:42:55.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:55.468+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:55.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:55.468+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:55.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:55.468+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:55.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:55.468+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:55.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:55 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:55.468+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:42:56.276 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:55.898+0000 7fddc1691700 -1 osd.8 103 *** Got signal Terminated *** 2024-04-21T10:42:56.276 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:55 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:55.898+0000 7fddc1691700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T10:42:56.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:55 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:55.989+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:56.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:55 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:55.989+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:56.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:55 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:55.989+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:56.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:55 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:55.989+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:56.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:55 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:55.989+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:42:56.776 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:56 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:56.456+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:56.777 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:56 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:56.456+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:56.777 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:56 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:56.456+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:56.777 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:56 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:56.456+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.753770+0000 front 2024-04-21T10:42:28.753878+0000 (oldest deadline 2024-04-21T10:42:51.053718+0000) 2024-04-21T10:42:56.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:56 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:56.295+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:56.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:56 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:56.295+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:56.778 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:56 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:56.295+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:56.778 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:56 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:56.295+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:56.778 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:56 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:56.295+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:42:56.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:56 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:56.449+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:56.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:56 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:56.449+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:56.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:56 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:56.449+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:56.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:56 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:56.449+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:56.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:56 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:56.449+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:42:57.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:56 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:56.999+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:57.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:56 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:56.999+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:57.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:56 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:56.999+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:57.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:56 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:56.999+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:57.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:56 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:56.999+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:42:57.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:57 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:57.487+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:57.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:57 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:57.487+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:57.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:57 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:57.487+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:57.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:57 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:57.487+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.753770+0000 front 2024-04-21T10:42:28.753878+0000 (oldest deadline 2024-04-21T10:42:51.053718+0000) 2024-04-21T10:42:57.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:57 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:57.257+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:57.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:57 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:57.257+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:57.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:57 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:57.257+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:57.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:57 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:57.257+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:57.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:57 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:57.257+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:42:57.856 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:57 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:57.484+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:57.856 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:57 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:57.484+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:57.856 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:57 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:57.484+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:57.856 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:57 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:57.484+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:57.856 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:57 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:57.484+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:42:58.447 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:57 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:57.962+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:58.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:57 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:57.962+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:58.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:57 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:57.962+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:58.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:57 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:57.962+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:58.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:57 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:57.962+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:42:58.526 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:58 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:58.455+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:58.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:58 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:58.455+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:58.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:58 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:58.455+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:58.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:58 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:58.455+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.753770+0000 front 2024-04-21T10:42:28.753878+0000 (oldest deadline 2024-04-21T10:42:51.053718+0000) 2024-04-21T10:42:58.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:58 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:58.455+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.655174+0000 front 2024-04-21T10:42:35.655192+0000 (oldest deadline 2024-04-21T10:42:57.955247+0000) 2024-04-21T10:42:58.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:58 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:58.283+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:58.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:58 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:58.283+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:58.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:58 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:58.283+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:58.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:58 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:58.283+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:58.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:58 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:58.283+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:42:58.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:58 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:58.283+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:42:58.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:58.477+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:58.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:58.477+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:58.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:58.477+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:58.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:58.477+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:58.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:58 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:58.477+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:42:59.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:58 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:58.981+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:42:59.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:58 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:58.981+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:42:59.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:58 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:58.981+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:42:59.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:58 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:58.981+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:42:59.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:42:58 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:42:58.981+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:42:59.776 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:59 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:59.292+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:42:59.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:59 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:59.292+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:42:59.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:59 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:59.292+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:42:59.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:59 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:59.292+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:42:59.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:59 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:59.292+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:42:59.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:42:59 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:42:59.292+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:42:59.777 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:59 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:59.454+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:42:59.778 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:59 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:59.454+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:42:59.778 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:59 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:59.454+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:42:59.778 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:59 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:59.454+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.753770+0000 front 2024-04-21T10:42:28.753878+0000 (oldest deadline 2024-04-21T10:42:51.053718+0000) 2024-04-21T10:42:59.778 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:42:59 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:42:59.454+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.655174+0000 front 2024-04-21T10:42:35.655192+0000 (oldest deadline 2024-04-21T10:42:57.955247+0000) 2024-04-21T10:42:59.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:59 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:59.491+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:42:59.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:59 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:59.491+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:42:59.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:59 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:59.491+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:42:59.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:59 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:59.491+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:42:59.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:59 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:59.491+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:42:59.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:42:59 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:42:59.491+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.182143+0000 front 2024-04-21T10:42:35.182171+0000 (oldest deadline 2024-04-21T10:42:59.282025+0000) 2024-04-21T10:43:00.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:00 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:00.029+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:00.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:00 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:00.029+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:00.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:00 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:00.029+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:00.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:00 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:00.029+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:00.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:00 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:00.029+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:00.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:00 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:00.029+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:00.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:00 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:00.275+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:00.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:00 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:00.275+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:00.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:00 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:00.275+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:00.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:00 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:00.275+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:00.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:00 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:00.275+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:00.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:00 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:00.275+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:00.527 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:00 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:43:00.472+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.850007+0000 front 2024-04-21T10:42:04.850090+0000 (oldest deadline 2024-04-21T10:42:26.550088+0000) 2024-04-21T10:43:00.528 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:00 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:43:00.472+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.851178+0000 front 2024-04-21T10:42:11.851223+0000 (oldest deadline 2024-04-21T10:42:35.351287+0000) 2024-04-21T10:43:00.528 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:00 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:43:00.472+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:21.152905+0000 front 2024-04-21T10:42:21.152944+0000 (oldest deadline 2024-04-21T10:42:47.052800+0000) 2024-04-21T10:43:00.528 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:00 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:43:00.472+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.753770+0000 front 2024-04-21T10:42:28.753878+0000 (oldest deadline 2024-04-21T10:42:51.053718+0000) 2024-04-21T10:43:00.528 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:00 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8[50052]: 2024-04-21T10:43:00.472+0000 7fddbdc85700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.655174+0000 front 2024-04-21T10:42:35.655192+0000 (oldest deadline 2024-04-21T10:42:57.955247+0000) 2024-04-21T10:43:00.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:00.459+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:43:00.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:00.459+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:43:00.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:00.459+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:43:00.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:00.459+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:43:00.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:00.459+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:43:00.936 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:00 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:00.459+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.182143+0000 front 2024-04-21T10:42:35.182171+0000 (oldest deadline 2024-04-21T10:42:59.282025+0000) 2024-04-21T10:43:01.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:01 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:01.066+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:01.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:01 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:01.066+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:01.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:01 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:01.066+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:01.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:01 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:01.066+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:01.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:01 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:01.066+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:01.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:01 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:01.066+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:01.673 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:01 smithi190 podman[68973]: 2024-04-21 10:43:01.344300028 +0000 UTC m=+5.552696190 container died 792ef178830c647c3a7457dceb11df213b51816f627563eb812ce608b6a2ec12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-21T10:43:01.674 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:01 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:01.291+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:01.674 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:01 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:01.291+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:01.674 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:01 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:01.291+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:01.674 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:01 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:01.291+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:01.674 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:01 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:01.291+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:01.674 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:01 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:01.291+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:01.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:01 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:01.483+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:43:01.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:01 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:01.483+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:43:01.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:01 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:01.483+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:43:01.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:01 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:01.483+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:43:01.936 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:01 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:01.483+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:43:01.936 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:01 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:01.483+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.182143+0000 front 2024-04-21T10:42:35.182171+0000 (oldest deadline 2024-04-21T10:42:59.282025+0000) 2024-04-21T10:43:01.981 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:01 smithi190 podman[68973]: 2024-04-21 10:43:01.757000828 +0000 UTC m=+5.965396983 container cleanup 792ef178830c647c3a7457dceb11df213b51816f627563eb812ce608b6a2ec12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-21T10:43:01.981 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:01 smithi190 bash[68973]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8 2024-04-21T10:43:02.276 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:01 smithi190 podman[68991]: 2024-04-21 10:43:01.982966443 +0000 UTC m=+0.630306504 container remove 792ef178830c647c3a7457dceb11df213b51816f627563eb812ce608b6a2ec12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-21T10:43:02.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:02 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:02.046+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:02.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:02 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:02.046+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:02.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:02 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:02.046+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:02.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:02 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:02.046+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:02.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:02 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:02.046+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:02.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:02 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:02.046+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:02.626 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:02 smithi190 podman[69079]: 2024-04-21 10:43:02.284699928 +0000 UTC m=+0.018719474 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:43:02.626 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:02 smithi190 podman[69079]: 2024-04-21 10:43:02.383474327 +0000 UTC m=+0.117493867 container create 5f0c2750d915c524a0c8b70f6ba672e16243d7cf59c9068548346f629980a937 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True) 2024-04-21T10:43:02.626 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:02 smithi190 podman[69079]: 2024-04-21 10:43:02.627369632 +0000 UTC m=+0.361389172 container init 5f0c2750d915c524a0c8b70f6ba672e16243d7cf59c9068548346f629980a937 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-21T10:43:02.627 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:02 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:02.282+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:02.627 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:02 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:02.282+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:02.627 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:02 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:02.282+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:02.627 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:02 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:02.282+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:02.627 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:02 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:02.282+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:02.627 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:02 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:02.282+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:02.921 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:02 smithi190 podman[69079]: 2024-04-21 10:43:02.630835633 +0000 UTC m=+0.364855179 container start 5f0c2750d915c524a0c8b70f6ba672e16243d7cf59c9068548346f629980a937 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-04-21T10:43:02.921 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:02 smithi190 podman[69079]: 2024-04-21 10:43:02.719033248 +0000 UTC m=+0.453052788 container attach 5f0c2750d915c524a0c8b70f6ba672e16243d7cf59c9068548346f629980a937 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-21T10:43:02.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:02 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:02.474+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:43:02.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:02 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:02.474+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:43:02.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:02 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:02.474+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:43:02.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:02 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:02.474+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:43:02.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:02 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:02.474+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:43:02.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:02 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:02.474+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.182143+0000 front 2024-04-21T10:42:35.182171+0000 (oldest deadline 2024-04-21T10:42:59.282025+0000) 2024-04-21T10:43:03.195 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:02 smithi190 podman[69079]: 2024-04-21 10:43:02.92323273 +0000 UTC m=+0.657252278 container died 5f0c2750d915c524a0c8b70f6ba672e16243d7cf59c9068548346f629980a937 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-21T10:43:03.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:03 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:03.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:03.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:03 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:03.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:03.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:03 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:03.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:03.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:03 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:03.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:03.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:03 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:03.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:03.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:03 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:03.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:03.454 INFO:journalctl@ceph.osd.8.smithi190.stdout:Apr 21 10:43:03 smithi190 podman[69105]: 2024-04-21 10:43:03.438617351 +0000 UTC m=+0.509868062 container remove 5f0c2750d915c524a0c8b70f6ba672e16243d7cf59c9068548346f629980a937 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:43:03.454 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:03 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:03.297+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:03.454 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:03 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:03.297+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:03.454 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:03 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:03.297+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:03.455 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:03 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:03.297+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:03.455 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:03 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:03.297+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:03.455 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:03 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:03.297+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:03.459 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.8.service' 2024-04-21T10:43:03.498 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:43:03.498 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-21T10:43:03.498 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-21T10:43:03.498 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.9 2024-04-21T10:43:03.913 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:03 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:03.483+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:43:03.913 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:03 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:03.483+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:43:03.913 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:03 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:03.483+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:43:03.914 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:03 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:03.483+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:43:03.914 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:03 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:03.483+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:43:03.914 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:03 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:03.483+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.182143+0000 front 2024-04-21T10:42:35.182171+0000 (oldest deadline 2024-04-21T10:42:59.282025+0000) 2024-04-21T10:43:03.914 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:03 smithi026 systemd[1]: Stopping Ceph osd.9 for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:43:03.914 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:03 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:03.913+0000 7f5b399ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:43:03.914 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:03 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:03.913+0000 7f5b399ad700 -1 osd.9 103 *** Got signal Terminated *** 2024-04-21T10:43:03.914 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:03 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:03.913+0000 7f5b399ad700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T10:43:04.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:04 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:04.124+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:04.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:04 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:04.124+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:04.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:04 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:04.124+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:04.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:04 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:04.124+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:04.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:04 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:04.124+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:04.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:04 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:04.124+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:04.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:04 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:04.250+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:04.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:04 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:04.250+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:04.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:04 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:04.250+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:04.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:04 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:04.250+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:04.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:04 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:04.250+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:04.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:04 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:04.250+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:04.934 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:04 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:04.487+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:43:04.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:04 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:04.487+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:43:04.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:04 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:04.487+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:43:04.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:04 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:04.487+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:43:04.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:04 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:04.487+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:43:04.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:04 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:04.487+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.182143+0000 front 2024-04-21T10:42:35.182171+0000 (oldest deadline 2024-04-21T10:42:59.282025+0000) 2024-04-21T10:43:05.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:05 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:05.080+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:05.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:05 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:05.080+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:05.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:05 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:05.080+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:05.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:05 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:05.080+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:05.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:05 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:05.080+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:05.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:05 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:05.080+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:05.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:05 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:05.206+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:05.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:05 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:05.206+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:05.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:05 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:05.206+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:05.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:05 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:05.206+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:05.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:05 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:05.206+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:05.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:05 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:05.206+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:05.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:05 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:05.442+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:43:05.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:05 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:05.442+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:43:05.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:05 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:05.442+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:43:05.935 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:05 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:05.442+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:43:05.936 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:05 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:05.442+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:43:05.936 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:05 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:05.442+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.182143+0000 front 2024-04-21T10:42:35.182171+0000 (oldest deadline 2024-04-21T10:42:59.282025+0000) 2024-04-21T10:43:06.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:06 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:06.094+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:06.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:06 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:06.094+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:06.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:06 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:06.094+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:06.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:06 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:06.094+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:06.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:06 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:06.094+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:06.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:06 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:06.094+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:06.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:06 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:06.224+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:06.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:06 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:06.224+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:06.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:06 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:06.224+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:06.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:06 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:06.224+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:06.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:06 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:06.224+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:06.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:06 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:06.224+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:06.684 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:06 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:06.413+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:43:06.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:06 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:06.413+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:43:06.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:06 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:06.413+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:43:06.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:06 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:06.413+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:43:06.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:06 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:06.413+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:43:06.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:06 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:06.413+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.182143+0000 front 2024-04-21T10:42:35.182171+0000 (oldest deadline 2024-04-21T10:42:59.282025+0000) 2024-04-21T10:43:07.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:07 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:07.071+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:07.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:07 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:07.071+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:07.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:07 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:07.071+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:07.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:07 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:07.071+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:07.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:07 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:07.071+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:07.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:07 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:07.071+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:07.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:07 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:07.244+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:07.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:07 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:07.244+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:07.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:07 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:07.244+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:07.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:07 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:07.244+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:07.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:07 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:07.244+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:07.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:07 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:07.244+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:07.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:07 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:07.422+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:43:07.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:07 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:07.422+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:43:07.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:07 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:07.422+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:43:07.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:07 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:07.422+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:43:07.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:07 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:07.422+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:43:07.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:07 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:07.422+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.182143+0000 front 2024-04-21T10:42:35.182171+0000 (oldest deadline 2024-04-21T10:42:59.282025+0000) 2024-04-21T10:43:08.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:08 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:08.026+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:08.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:08 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:08.026+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:08.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:08 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:08.026+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:08.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:08 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:08.026+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:08.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:08 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:08.026+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:08.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:08 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:08.026+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:08.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:08 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:08.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:08.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:08 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:08.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:08.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:08 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:08.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:08.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:08 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:08.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:08.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:08 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:08.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:08.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:08 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:08.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:08.684 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:08 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:08.411+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.474612+0000 front 2024-04-21T10:41:54.474650+0000 (oldest deadline 2024-04-21T10:42:19.774807+0000) 2024-04-21T10:43:08.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:08 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:08.411+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.876014+0000 front 2024-04-21T10:42:03.876078+0000 (oldest deadline 2024-04-21T10:42:27.876733+0000) 2024-04-21T10:43:08.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:08 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:08.411+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.378353+0000 front 2024-04-21T10:42:11.378198+0000 (oldest deadline 2024-04-21T10:42:38.378410+0000) 2024-04-21T10:43:08.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:08 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:08.411+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.378892+0000 front 2024-04-21T10:42:18.379067+0000 (oldest deadline 2024-04-21T10:42:42.479146+0000) 2024-04-21T10:43:08.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:08 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:08.411+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.181312+0000 front 2024-04-21T10:42:28.181289+0000 (oldest deadline 2024-04-21T10:42:52.880995+0000) 2024-04-21T10:43:08.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:08 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9[60019]: 2024-04-21T10:43:08.411+0000 7f5b35fa1700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:35.182143+0000 front 2024-04-21T10:42:35.182171+0000 (oldest deadline 2024-04-21T10:42:59.282025+0000) 2024-04-21T10:43:09.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:09 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:09.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:09.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:09 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:09.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:09.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:09 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:09.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:09.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:09 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:09.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:09.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:09 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:09.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:09.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:09 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:09.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:09.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:09 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:09.230+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:09.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:09 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:09.230+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:09.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:09 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:09.230+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:09.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:09 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:09.230+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:09.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:09 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:09.230+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:09.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:09 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:09.230+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:09.684 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:09 smithi026 podman[91766]: 2024-04-21 10:43:09.341796647 +0000 UTC m=+5.641417762 container died 2fb66ec07df08ec3805f8e4bc68e0a6fd99c629e9f633f19cc96cb79cc0fdd54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:43:10.004 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:09 smithi026 podman[91766]: 2024-04-21 10:43:09.788529111 +0000 UTC m=+6.088150237 container cleanup 2fb66ec07df08ec3805f8e4bc68e0a6fd99c629e9f633f19cc96cb79cc0fdd54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-21T10:43:10.004 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:09 smithi026 bash[91766]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9 2024-04-21T10:43:10.413 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:10 smithi026 podman[91784]: 2024-04-21 10:43:10.006102405 +0000 UTC m=+0.658240278 container remove 2fb66ec07df08ec3805f8e4bc68e0a6fd99c629e9f633f19cc96cb79cc0fdd54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS) 2024-04-21T10:43:10.447 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:10 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:10.095+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:10.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:10 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:10.095+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:10.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:10 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:10.095+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:10.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:10 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:10.095+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:10.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:10 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:10.095+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:10.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:10 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:10.095+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:10.529 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:10 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:10.233+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:10.530 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:10 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:10.233+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:10.531 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:10 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:10.233+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:10.531 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:10 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:10.233+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:10.531 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:10 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:10.233+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:10.531 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:10 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:10.233+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:10.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:10 smithi026 podman[91877]: 2024-04-21 10:43:10.415277561 +0000 UTC m=+0.114139388 container create 240dd31ea294b315c1c41ff9750423f88e8ab22069255dd9d0ca1442eacd1f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-21T10:43:10.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:10 smithi026 podman[91877]: 2024-04-21 10:43:10.319783982 +0000 UTC m=+0.018645787 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:43:10.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:10 smithi026 podman[91877]: 2024-04-21 10:43:10.617333551 +0000 UTC m=+0.316195337 container init 240dd31ea294b315c1c41ff9750423f88e8ab22069255dd9d0ca1442eacd1f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-21T10:43:10.685 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:10 smithi026 podman[91877]: 2024-04-21 10:43:10.620883454 +0000 UTC m=+0.319745245 container start 240dd31ea294b315c1c41ff9750423f88e8ab22069255dd9d0ca1442eacd1f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-04-21T10:43:11.185 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:10 smithi026 podman[91877]: 2024-04-21 10:43:10.708846588 +0000 UTC m=+0.407708379 container attach 240dd31ea294b315c1c41ff9750423f88e8ab22069255dd9d0ca1442eacd1f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-04-21T10:43:11.185 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:10 smithi026 podman[91877]: 2024-04-21 10:43:10.930162342 +0000 UTC m=+0.629024142 container died 240dd31ea294b315c1c41ff9750423f88e8ab22069255dd9d0ca1442eacd1f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-21T10:43:11.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:11 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:11.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:11.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:11 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:11.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:11.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:11 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:11.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:11.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:11 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:11.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:11.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:11 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:11.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:11.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:11 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:11.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:11.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:11 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:11.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-04-21T10:42:44.488965+0000 front 2024-04-21T10:42:44.488903+0000 (oldest deadline 2024-04-21T10:43:10.388840+0000) 2024-04-21T10:43:11.474 INFO:journalctl@ceph.osd.9.smithi026.stdout:Apr 21 10:43:11 smithi026 podman[91904]: 2024-04-21 10:43:11.453690137 +0000 UTC m=+0.518282593 container remove 240dd31ea294b315c1c41ff9750423f88e8ab22069255dd9d0ca1442eacd1f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-21T10:43:11.479 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.9.service' 2024-04-21T10:43:11.517 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:43:11.518 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-21T10:43:11.518 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-21T10:43:11.518 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.10 2024-04-21T10:43:11.776 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:11 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:11.279+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:11.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:11 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:11.279+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:11.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:11 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:11.279+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:11.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:11 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:11.279+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:11.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:11 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:11.279+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:11.777 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:11 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:11.279+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:11.813 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:11 smithi169 systemd[1]: Stopping Ceph osd.10 for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:43:12.096 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:11 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:11.812+0000 7f3d0d522700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:43:12.096 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:11 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:11.812+0000 7f3d0d522700 -1 osd.10 103 *** Got signal Terminated *** 2024-04-21T10:43:12.097 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:11 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:11.812+0000 7f3d0d522700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T10:43:12.097 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:12 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:12.097+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:12.097 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:12 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:12.097+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:12.097 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:12 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:12.097+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:12.097 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:12 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:12.097+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:12.097 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:12 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:12.097+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:12.447 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:12 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:12.097+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:12.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:12 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:12.097+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-04-21T10:42:44.488965+0000 front 2024-04-21T10:42:44.488903+0000 (oldest deadline 2024-04-21T10:43:10.388840+0000) 2024-04-21T10:43:12.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:12 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:12.249+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:12.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:12 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:12.249+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:12.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:12 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:12.249+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:12.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:12 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:12.249+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:12.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:12 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:12.249+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:12.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:12 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:12.249+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:13.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:13 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:13.063+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:13.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:13 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:13.063+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:13.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:13 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:13.063+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:13.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:13 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:13.063+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:13.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:13 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:13.063+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:13.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:13 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:13.063+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:13.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:13 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:13.063+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-04-21T10:42:44.488965+0000 front 2024-04-21T10:42:44.488903+0000 (oldest deadline 2024-04-21T10:43:10.388840+0000) 2024-04-21T10:43:13.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:13 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:13.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:13.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:13 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:13.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:13.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:13 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:13.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:13.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:13 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:13.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:13.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:13 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:13.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:13.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:13 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:13.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:14.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:14.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:14.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:14.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:14.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:14.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:14.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:14.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:14.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:14.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:14.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:14.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:14.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:14 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:14.068+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-04-21T10:42:44.488965+0000 front 2024-04-21T10:42:44.488903+0000 (oldest deadline 2024-04-21T10:43:10.388840+0000) 2024-04-21T10:43:14.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:14 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:14.217+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:14.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:14 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:14.217+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:14.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:14 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:14.217+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:14.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:14 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:14.217+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:14.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:14 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:14.217+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:14.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:14 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:14.217+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:15.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:15 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:15.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:15.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:15 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:15.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:15.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:15 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:15.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:15.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:15 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:15.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:15.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:15 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:15.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:15.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:15 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:15.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:15.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:15 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:15.103+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-04-21T10:42:44.488965+0000 front 2024-04-21T10:42:44.488903+0000 (oldest deadline 2024-04-21T10:43:10.388840+0000) 2024-04-21T10:43:15.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:15 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:15.207+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:15.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:15 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:15.207+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:15.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:15 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:15.207+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:15.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:15 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:15.207+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:15.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:15 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:15.207+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:15.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:15 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:15.207+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:16.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:16 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:16.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:55.077351+0000 front 2024-04-21T10:41:55.077434+0000 (oldest deadline 2024-04-21T10:42:19.777364+0000) 2024-04-21T10:43:16.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:16 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:16.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:04.779936+0000 front 2024-04-21T10:42:04.779961+0000 (oldest deadline 2024-04-21T10:42:26.380768+0000) 2024-04-21T10:43:16.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:16 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:16.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:13.282628+0000 front 2024-04-21T10:42:13.282733+0000 (oldest deadline 2024-04-21T10:42:34.382722+0000) 2024-04-21T10:43:16.448 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:16 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:16.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:18.984304+0000 front 2024-04-21T10:42:18.984468+0000 (oldest deadline 2024-04-21T10:42:41.884315+0000) 2024-04-21T10:43:16.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:16 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:16.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:28.285777+0000 front 2024-04-21T10:42:28.285722+0000 (oldest deadline 2024-04-21T10:42:50.585858+0000) 2024-04-21T10:43:16.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:16 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:16.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.387037+0000 front 2024-04-21T10:42:33.387145+0000 (oldest deadline 2024-04-21T10:42:59.287084+0000) 2024-04-21T10:43:16.449 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:16 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10[56641]: 2024-04-21T10:43:16.090+0000 7f3d09b16700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2024-04-21T10:42:44.488965+0000 front 2024-04-21T10:42:44.488903+0000 (oldest deadline 2024-04-21T10:43:10.388840+0000) 2024-04-21T10:43:16.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:16 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:16.247+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:16.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:16 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:16.247+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:16.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:16 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:16.247+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:16.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:16 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:16.247+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:16.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:16 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:16.247+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:16.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:16 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:16.247+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:17.405 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:17 smithi169 podman[68457]: 2024-04-21 10:43:17.152412562 +0000 UTC m=+5.438275662 container died 25fc627952ebb3eb954ac9ee37f9cd0fde8059865f6825fdba2dd0662ea4c8bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-21T10:43:17.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:17 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:17.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:17.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:17 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:17.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:17.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:17 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:17.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:17.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:17 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:17.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:17.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:17 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:17.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:17.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:17 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:17.260+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:17.680 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:17 smithi169 podman[68457]: 2024-04-21 10:43:17.488598479 +0000 UTC m=+5.774461579 container cleanup 25fc627952ebb3eb954ac9ee37f9cd0fde8059865f6825fdba2dd0662ea4c8bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-21T10:43:17.680 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:17 smithi169 bash[68457]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10 2024-04-21T10:43:17.947 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:17 smithi169 podman[68474]: 2024-04-21 10:43:17.681409102 +0000 UTC m=+0.526267305 container remove 25fc627952ebb3eb954ac9ee37f9cd0fde8059865f6825fdba2dd0662ea4c8bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-21T10:43:18.407 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:18 smithi169 podman[68566]: 2024-04-21 10:43:18.008317456 +0000 UTC m=+0.018185665 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:43:18.408 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:18 smithi169 podman[68566]: 2024-04-21 10:43:18.115527605 +0000 UTC m=+0.125395805 container create bd86e9c9d35505e85a86b25bbe8ff084d3ffb2e27e33a03bb35a7116724e39f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0) 2024-04-21T10:43:18.408 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:18 smithi169 podman[68566]: 2024-04-21 10:43:18.342786926 +0000 UTC m=+0.352655130 container init bd86e9c9d35505e85a86b25bbe8ff084d3ffb2e27e33a03bb35a7116724e39f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:43:18.408 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:18 smithi169 podman[68566]: 2024-04-21 10:43:18.345821689 +0000 UTC m=+0.355689890 container start bd86e9c9d35505e85a86b25bbe8ff084d3ffb2e27e33a03bb35a7116724e39f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-21T10:43:18.408 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:18 smithi169 podman[68566]: 2024-04-21 10:43:18.409348772 +0000 UTC m=+0.419216986 container attach bd86e9c9d35505e85a86b25bbe8ff084d3ffb2e27e33a03bb35a7116724e39f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-21T10:43:18.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:18 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:18.243+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:18.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:18 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:18.243+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:18.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:18 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:18.243+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:18.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:18 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:18.243+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:18.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:18 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:18.243+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:18.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:18 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:18.243+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:18.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:18 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:18.243+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-04-21T10:42:52.865004+0000 front 2024-04-21T10:42:52.864982+0000 (oldest deadline 2024-04-21T10:43:17.564811+0000) 2024-04-21T10:43:18.697 INFO:journalctl@ceph.osd.10.smithi169.stdout:Apr 21 10:43:18 smithi169 podman[68566]: 2024-04-21 10:43:18.639829175 +0000 UTC m=+0.649697367 container died bd86e9c9d35505e85a86b25bbe8ff084d3ffb2e27e33a03bb35a7116724e39f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-21T10:43:19.206 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.10.service' 2024-04-21T10:43:19.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:43:19.243 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-21T10:43:19.243 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-21T10:43:19.243 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.11 2024-04-21T10:43:19.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:19 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:19.202+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:19.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:19 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:19.202+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:19.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:19 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:19.202+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:19.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:19 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:19.202+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:19.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:19 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:19.202+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:19.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:19 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:19.202+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:19.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:19 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:19.202+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-04-21T10:42:52.865004+0000 front 2024-04-21T10:42:52.864982+0000 (oldest deadline 2024-04-21T10:43:17.564811+0000) 2024-04-21T10:43:19.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:19 smithi190 systemd[1]: Stopping Ceph osd.11 for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:43:20.026 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:19 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:19.591+0000 7f81f4f92700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:43:20.027 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:19 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:19.591+0000 7f81f4f92700 -1 osd.11 103 *** Got signal Terminated *** 2024-04-21T10:43:20.027 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:19 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:19.591+0000 7f81f4f92700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T10:43:20.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:20 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:20.235+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:20.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:20 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:20.235+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:20.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:20 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:20.235+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:20.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:20 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:20.235+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:20.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:20 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:20.235+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:20.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:20 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:20.235+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:20.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:20 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:20.235+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-04-21T10:42:52.865004+0000 front 2024-04-21T10:42:52.864982+0000 (oldest deadline 2024-04-21T10:43:17.564811+0000) 2024-04-21T10:43:21.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:21 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:21.196+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:21.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:21 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:21.196+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:21.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:21 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:21.196+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:21.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:21 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:21.196+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:21.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:21 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:21.196+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:21.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:21 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:21.196+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:21.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:21 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:21.196+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-04-21T10:42:52.865004+0000 front 2024-04-21T10:42:52.864982+0000 (oldest deadline 2024-04-21T10:43:17.564811+0000) 2024-04-21T10:43:22.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:22 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:22.195+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:22.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:22 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:22.195+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:22.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:22 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:22.195+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:22.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:22 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:22.195+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:22.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:22 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:22.195+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:22.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:22 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:22.195+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:22.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:22 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:22.195+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-04-21T10:42:52.865004+0000 front 2024-04-21T10:42:52.864982+0000 (oldest deadline 2024-04-21T10:43:17.564811+0000) 2024-04-21T10:43:23.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:23 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:23.149+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:23.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:23 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:23.149+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:23.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:23 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:23.149+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:23.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:23 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:23.149+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:23.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:23 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:23.149+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:23.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:23 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:23.149+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:23.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:23 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:23.149+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-04-21T10:42:52.865004+0000 front 2024-04-21T10:42:52.864982+0000 (oldest deadline 2024-04-21T10:43:17.564811+0000) 2024-04-21T10:43:23.528 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:23 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:23.149+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6820 osd.8 since back 2024-04-21T10:42:57.565143+0000 front 2024-04-21T10:42:57.565164+0000 (oldest deadline 2024-04-21T10:43:22.865184+0000) 2024-04-21T10:43:24.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:24 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:24.178+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2024-04-21T10:41:54.556483+0000 front 2024-04-21T10:41:54.556458+0000 (oldest deadline 2024-04-21T10:42:18.656610+0000) 2024-04-21T10:43:24.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:24 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:24.178+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6804 osd.1 since back 2024-04-21T10:42:03.858056+0000 front 2024-04-21T10:42:03.858162+0000 (oldest deadline 2024-04-21T10:42:28.558198+0000) 2024-04-21T10:43:24.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:24 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:24.178+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2024-04-21T10:42:11.459098+0000 front 2024-04-21T10:42:11.458976+0000 (oldest deadline 2024-04-21T10:42:35.559148+0000) 2024-04-21T10:43:24.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:24 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:24.178+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2024-04-21T10:42:17.860262+0000 front 2024-04-21T10:42:17.860332+0000 (oldest deadline 2024-04-21T10:42:43.760329+0000) 2024-04-21T10:43:24.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:24 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:24.178+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6812 osd.4 since back 2024-04-21T10:42:23.760953+0000 front 2024-04-21T10:42:23.760902+0000 (oldest deadline 2024-04-21T10:42:53.161219+0000) 2024-04-21T10:43:24.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:24 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:24.178+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2024-04-21T10:42:33.161557+0000 front 2024-04-21T10:42:33.161658+0000 (oldest deadline 2024-04-21T10:42:57.861805+0000) 2024-04-21T10:43:24.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:24 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:24.178+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.169:6820 osd.7 since back 2024-04-21T10:42:52.865004+0000 front 2024-04-21T10:42:52.864982+0000 (oldest deadline 2024-04-21T10:43:17.564811+0000) 2024-04-21T10:43:24.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:24 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11[57109]: 2024-04-21T10:43:24.178+0000 7f81ed57e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.190:6820 osd.8 since back 2024-04-21T10:42:57.565143+0000 front 2024-04-21T10:42:57.565164+0000 (oldest deadline 2024-04-21T10:43:22.865184+0000) 2024-04-21T10:43:25.256 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:24 smithi190 podman[69209]: 2024-04-21 10:43:24.978583435 +0000 UTC m=+5.553465485 container died 6429ceabb85c19e456a9f3b878899ca00179b589d0fafb243bd30e3c21da7f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-21T10:43:25.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:25 smithi190 podman[69209]: 2024-04-21 10:43:25.365743051 +0000 UTC m=+5.940625105 container cleanup 6429ceabb85c19e456a9f3b878899ca00179b589d0fafb243bd30e3c21da7f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-04-21T10:43:25.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:25 smithi190 bash[69209]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11 2024-04-21T10:43:25.983 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:25 smithi190 podman[69225]: 2024-04-21 10:43:25.625210804 +0000 UTC m=+0.647119220 container remove 6429ceabb85c19e456a9f3b878899ca00179b589d0fafb243bd30e3c21da7f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-04-21T10:43:26.243 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:25 smithi190 podman[69316]: 2024-04-21 10:43:25.888326334 +0000 UTC m=+0.018137965 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T10:43:26.243 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:26 smithi190 podman[69316]: 2024-04-21 10:43:26.00100852 +0000 UTC m=+0.130820140 container create 2fa75149cfbeaf863ec0f4e6759477d51e0b3245edc45de18acef9a178340db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-21T10:43:26.244 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:26 smithi190 podman[69316]: 2024-04-21 10:43:26.245233642 +0000 UTC m=+0.375045263 container init 2fa75149cfbeaf863ec0f4e6759477d51e0b3245edc45de18acef9a178340db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-21T10:43:26.526 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:26 smithi190 podman[69316]: 2024-04-21 10:43:26.248305501 +0000 UTC m=+0.378117123 container start 2fa75149cfbeaf863ec0f4e6759477d51e0b3245edc45de18acef9a178340db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:43:26.527 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:26 smithi190 podman[69316]: 2024-04-21 10:43:26.32832021 +0000 UTC m=+0.458131841 container attach 2fa75149cfbeaf863ec0f4e6759477d51e0b3245edc45de18acef9a178340db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T10:43:26.821 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:26 smithi190 podman[69316]: 2024-04-21 10:43:26.533383551 +0000 UTC m=+0.663195171 container died 2fa75149cfbeaf863ec0f4e6759477d51e0b3245edc45de18acef9a178340db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:43:27.074 INFO:journalctl@ceph.osd.11.smithi190.stdout:Apr 21 10:43:27 smithi190 podman[69342]: 2024-04-21 10:43:27.072916078 +0000 UTC m=+0.531007325 container remove 2fa75149cfbeaf863ec0f4e6759477d51e0b3245edc45de18acef9a178340db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-21T10:43:27.094 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@osd.11.service' 2024-04-21T10:43:27.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:43:27.134 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-21T10:43:27.134 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-21T10:43:27.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.a 2024-04-21T10:43:27.434 INFO:journalctl@ceph.mds.a.smithi026.stdout:Apr 21 10:43:27 smithi026 systemd[1]: Stopping Ceph mds.a for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:43:27.935 INFO:journalctl@ceph.mds.a.smithi026.stdout:Apr 21 10:43:27 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-a[67008]: 2024-04-21T10:43:27.438+0000 7ffa47f4a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:43:27.935 INFO:journalctl@ceph.mds.a.smithi026.stdout:Apr 21 10:43:27 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-a[67008]: 2024-04-21T10:43:27.438+0000 7ffa47f4a700 -1 mds.a *** got signal Terminated *** 2024-04-21T10:43:29.863 INFO:journalctl@ceph.mds.a.smithi026.stdout:Apr 21 10:43:29 smithi026 podman[91994]: 2024-04-21 10:43:29.595730733 +0000 UTC m=+2.259576061 container died 862ab8ba0c5b86dfd5c2638fe6d0fe94195a7c8f99dacb480119f15d5c373b43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T10:43:30.113 INFO:journalctl@ceph.mds.a.smithi026.stdout:Apr 21 10:43:29 smithi026 podman[91994]: 2024-04-21 10:43:29.938545333 +0000 UTC m=+2.602390659 container cleanup 862ab8ba0c5b86dfd5c2638fe6d0fe94195a7c8f99dacb480119f15d5c373b43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:43:30.113 INFO:journalctl@ceph.mds.a.smithi026.stdout:Apr 21 10:43:29 smithi026 bash[91994]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-a 2024-04-21T10:43:30.363 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.a.service' 2024-04-21T10:43:30.384 INFO:journalctl@ceph.mds.a.smithi026.stdout:Apr 21 10:43:30 smithi026 podman[92008]: 2024-04-21 10:43:30.114626987 +0000 UTC m=+0.516332534 container remove 862ab8ba0c5b86dfd5c2638fe6d0fe94195a7c8f99dacb480119f15d5c373b43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-21T10:43:30.384 INFO:journalctl@ceph.mds.a.smithi026.stdout:Apr 21 10:43:30 smithi026 systemd[1]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.a.service: Deactivated successfully. 2024-04-21T10:43:30.384 INFO:journalctl@ceph.mds.a.smithi026.stdout:Apr 21 10:43:30 smithi026 systemd[1]: Stopped Ceph mds.a for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:43:30.384 INFO:journalctl@ceph.mds.a.smithi026.stdout:Apr 21 10:43:30 smithi026 systemd[1]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.a.service: Consumed 1.110s CPU time. 2024-04-21T10:43:30.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:43:30.398 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-21T10:43:30.401 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-21T10:43:30.401 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.d 2024-04-21T10:43:30.685 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:43:30 smithi026 systemd[1]: Stopping Ceph mds.d for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:43:31.185 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:43:30 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-d[66042]: 2024-04-21T10:43:30.725+0000 7f3058550700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:43:31.185 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:43:30 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-d[66042]: 2024-04-21T10:43:30.725+0000 7f3058550700 -1 mds.d *** got signal Terminated *** 2024-04-21T10:43:34.540 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:43:34 smithi026 podman[92114]: 2024-04-21 10:43:34.29126213 +0000 UTC m=+3.672108999 container died 3675e1e3c4250ddb23e9752edfdbc46da32d3a571d2b828ba3fe99bb02224270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T10:43:34.540 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:43:34 smithi026 podman[92114]: 2024-04-21 10:43:34.541724063 +0000 UTC m=+3.922570932 container cleanup 3675e1e3c4250ddb23e9752edfdbc46da32d3a571d2b828ba3fe99bb02224270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-21T10:43:34.791 INFO:journalctl@ceph.mds.d.smithi026.stdout:Apr 21 10:43:34 smithi026 bash[92114]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-d 2024-04-21T10:43:34.936 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.d.service' 2024-04-21T10:43:34.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:43:34.970 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-21T10:43:34.970 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-21T10:43:34.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.g 2024-04-21T10:43:35.303 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:43:35 smithi026 systemd[1]: Stopping Ceph mds.g for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:43:35.684 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:43:35 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-g[66650]: 2024-04-21T10:43:35.304+0000 7f34cff9e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:43:35.685 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:43:35 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-g[66650]: 2024-04-21T10:43:35.304+0000 7f34cff9e700 -1 mds.g *** got signal Terminated *** 2024-04-21T10:43:39.161 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:43:38 smithi026 podman[92237]: 2024-04-21 10:43:38.89806563 +0000 UTC m=+3.704712174 container died 9494afbdcc141d2846ab7b680dd46c5fc42de262f43cdc14701e7d4b4579d74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T10:43:39.434 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:43:39 smithi026 podman[92237]: 2024-04-21 10:43:39.26197125 +0000 UTC m=+4.068617791 container cleanup 9494afbdcc141d2846ab7b680dd46c5fc42de262f43cdc14701e7d4b4579d74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-21T10:43:39.435 INFO:journalctl@ceph.mds.g.smithi026.stdout:Apr 21 10:43:39 smithi026 bash[92237]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-g 2024-04-21T10:43:39.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.g.service' 2024-04-21T10:43:39.790 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:43:39.790 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-21T10:43:39.791 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-21T10:43:39.791 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.j 2024-04-21T10:43:40.148 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:43:39 smithi026 systemd[1]: Stopping Ceph mds.j for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:43:40.434 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:43:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-j[65655]: 2024-04-21T10:43:40.148+0000 7fd2c686b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:43:40.435 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:43:40 smithi026 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-j[65655]: 2024-04-21T10:43:40.148+0000 7fd2c686b700 -1 mds.j *** got signal Terminated *** 2024-04-21T10:43:43.685 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:43:43 smithi026 podman[92358]: 2024-04-21 10:43:43.419413105 +0000 UTC m=+3.406086270 container died d71fb6b067d4baca0f0a039c151214409b3c098e179aadec700a77dcdea84a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-21T10:43:44.112 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:43:43 smithi026 podman[92358]: 2024-04-21 10:43:43.740374071 +0000 UTC m=+3.727047236 container cleanup d71fb6b067d4baca0f0a039c151214409b3c098e179aadec700a77dcdea84a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-j, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-21T10:43:44.112 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:43:43 smithi026 bash[92358]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-j 2024-04-21T10:43:44.113 INFO:journalctl@ceph.mds.j.smithi026.stdout:Apr 21 10:43:43 smithi026 podman[92372]: 2024-04-21 10:43:43.908291387 +0000 UTC m=+0.485075873 container remove d71fb6b067d4baca0f0a039c151214409b3c098e179aadec700a77dcdea84a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-21T10:43:44.137 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.j.service' 2024-04-21T10:43:44.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:43:44.169 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-21T10:43:44.169 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-21T10:43:44.169 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.b 2024-04-21T10:43:44.448 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:43:44 smithi169 systemd[1]: Stopping Ceph mds.b for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:43:44.948 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:43:44 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b[62141]: 2024-04-21T10:43:44.646+0000 7f7b5edcd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:43:44.948 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:43:44 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b[62141]: 2024-04-21T10:43:44.646+0000 7f7b5edcd700 -1 mds.b *** got signal Terminated *** 2024-04-21T10:43:48.929 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:43:48 smithi169 podman[68684]: 2024-04-21 10:43:48.654193324 +0000 UTC m=+4.287663459 container died 9b7427cf3b1152abb1082d43a31939579a00721a180ac49f4dbb87939ffa73d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-21T10:43:49.198 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:43:49 smithi169 podman[68684]: 2024-04-21 10:43:49.021687008 +0000 UTC m=+4.655157093 container cleanup 9b7427cf3b1152abb1082d43a31939579a00721a180ac49f4dbb87939ffa73d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-21T10:43:49.198 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:43:49 smithi169 bash[68684]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b 2024-04-21T10:43:49.452 INFO:journalctl@ceph.mds.b.smithi169.stdout:Apr 21 10:43:49 smithi169 podman[68698]: 2024-04-21 10:43:49.306498808 +0000 UTC m=+0.644251177 container remove 9b7427cf3b1152abb1082d43a31939579a00721a180ac49f4dbb87939ffa73d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-21T10:43:49.456 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.b.service' 2024-04-21T10:43:49.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:43:49.491 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-21T10:43:49.492 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-21T10:43:49.492 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.e 2024-04-21T10:43:49.900 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:43:49 smithi169 systemd[1]: Stopping Ceph mds.e for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:43:50.197 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:43:49 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e[61511]: 2024-04-21T10:43:49.899+0000 7f22fb999700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:43:50.198 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:43:49 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e[61511]: 2024-04-21T10:43:49.899+0000 7f22fb999700 -1 mds.e *** got signal Terminated *** 2024-04-21T10:43:53.808 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:43:53 smithi169 podman[68807]: 2024-04-21 10:43:53.537717155 +0000 UTC m=+3.830023347 container died ed196f30de18d8c8d3b9b1abdc55ab729674b87b61dd6c21924ec1b74dbff23f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-21T10:43:54.166 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:43:53 smithi169 podman[68807]: 2024-04-21 10:43:53.983918229 +0000 UTC m=+4.276224409 container cleanup ed196f30de18d8c8d3b9b1abdc55ab729674b87b61dd6c21924ec1b74dbff23f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-21T10:43:54.167 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:43:53 smithi169 bash[68807]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e 2024-04-21T10:43:54.403 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.e.service' 2024-04-21T10:43:54.424 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:43:54 smithi169 podman[68821]: 2024-04-21 10:43:54.168362896 +0000 UTC m=+0.624713930 container remove ed196f30de18d8c8d3b9b1abdc55ab729674b87b61dd6c21924ec1b74dbff23f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True) 2024-04-21T10:43:54.424 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:43:54 smithi169 systemd[1]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.e.service: Deactivated successfully. 2024-04-21T10:43:54.424 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:43:54 smithi169 systemd[1]: Stopped Ceph mds.e for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:43:54.425 INFO:journalctl@ceph.mds.e.smithi169.stdout:Apr 21 10:43:54 smithi169 systemd[1]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.e.service: Consumed 5.070s CPU time. 2024-04-21T10:43:54.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:43:54.436 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-21T10:43:54.437 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-21T10:43:54.437 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.h 2024-04-21T10:43:54.698 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:43:54 smithi169 systemd[1]: Stopping Ceph mds.h for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:43:55.198 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:43:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-h[61820]: 2024-04-21T10:43:54.777+0000 7f5fddb11700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:43:55.198 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:43:54 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-h[61820]: 2024-04-21T10:43:54.777+0000 7f5fddb11700 -1 mds.h *** got signal Terminated *** 2024-04-21T10:43:57.948 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:43:57 smithi169 podman[68926]: 2024-04-21 10:43:57.53573104 +0000 UTC m=+2.882381337 container died 80036354afb85f5d05f7f4f304ee005c523a984d405ba35efe4a1bfa1273328b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-21T10:43:58.447 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:43:58 smithi169 podman[68926]: 2024-04-21 10:43:58.195931482 +0000 UTC m=+3.542581773 container cleanup 80036354afb85f5d05f7f4f304ee005c523a984d405ba35efe4a1bfa1273328b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-21T10:43:58.448 INFO:journalctl@ceph.mds.h.smithi169.stdout:Apr 21 10:43:58 smithi169 bash[68926]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-h 2024-04-21T10:43:58.623 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.h.service' 2024-04-21T10:43:58.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:43:58.653 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-21T10:43:58.654 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-21T10:43:58.654 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.k 2024-04-21T10:43:58.981 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:43:58 smithi169 systemd[1]: Stopping Ceph mds.k for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:43:59.447 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:43:58 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-k[61205]: 2024-04-21T10:43:58.982+0000 7f7fec9cb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:43:59.448 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:43:58 smithi169 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-k[61205]: 2024-04-21T10:43:58.982+0000 7f7fec9cb700 -1 mds.k *** got signal Terminated *** 2024-04-21T10:44:00.948 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:44:00 smithi169 podman[69046]: 2024-04-21 10:44:00.673011331 +0000 UTC m=+1.800319422 container died 58f7521f39941eb78fb655960f436114e9d460e525233d3c485fe32a6a18728e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T10:44:01.319 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:44:00 smithi169 podman[69046]: 2024-04-21 10:44:00.966471437 +0000 UTC m=+2.093779536 container cleanup 58f7521f39941eb78fb655960f436114e9d460e525233d3c485fe32a6a18728e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-21T10:44:01.319 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:44:00 smithi169 bash[69046]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-k 2024-04-21T10:44:01.319 INFO:journalctl@ceph.mds.k.smithi169.stdout:Apr 21 10:44:01 smithi169 podman[69060]: 2024-04-21 10:44:01.19245501 +0000 UTC m=+0.510927360 container remove 58f7521f39941eb78fb655960f436114e9d460e525233d3c485fe32a6a18728e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-k, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-21T10:44:01.341 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.k.service' 2024-04-21T10:44:01.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:44:01.372 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-21T10:44:01.372 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-21T10:44:01.372 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.c 2024-04-21T10:44:01.688 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:44:01 smithi190 systemd[1]: Stopping Ceph mds.c for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:44:02.026 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:44:01 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c[62429]: 2024-04-21T10:44:01.688+0000 7f980711c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:44:02.026 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:44:01 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c[62429]: 2024-04-21T10:44:01.688+0000 7f980711c700 -1 mds.c *** got signal Terminated *** 2024-04-21T10:44:05.837 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:44:05 smithi190 podman[69424]: 2024-04-21 10:44:05.532931466 +0000 UTC m=+3.953790747 container died 945f9b9fd7f44de64a4ecf7e3fd56443c82af5178da400cd8fcab55d258a6f64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-21T10:44:06.121 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:44:05 smithi190 podman[69424]: 2024-04-21 10:44:05.938198058 +0000 UTC m=+4.359057337 container cleanup 945f9b9fd7f44de64a4ecf7e3fd56443c82af5178da400cd8fcab55d258a6f64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-21T10:44:06.121 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:44:05 smithi190 bash[69424]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c 2024-04-21T10:44:06.362 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.c.service' 2024-04-21T10:44:06.383 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:44:06 smithi190 podman[69449]: 2024-04-21 10:44:06.122643619 +0000 UTC m=+0.580088967 container remove 945f9b9fd7f44de64a4ecf7e3fd56443c82af5178da400cd8fcab55d258a6f64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-c, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-21T10:44:06.383 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:44:06 smithi190 systemd[1]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.c.service: Deactivated successfully. 2024-04-21T10:44:06.383 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:44:06 smithi190 systemd[1]: Stopped Ceph mds.c for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:44:06.383 INFO:journalctl@ceph.mds.c.smithi190.stdout:Apr 21 10:44:06 smithi190 systemd[1]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.c.service: Consumed 34.246s CPU time. 2024-04-21T10:44:06.397 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:44:06.397 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-21T10:44:06.397 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-21T10:44:06.397 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.f 2024-04-21T10:44:06.716 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:44:06 smithi190 systemd[1]: Stopping Ceph mds.f for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:44:07.026 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:44:06 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-f[62124]: 2024-04-21T10:44:06.716+0000 7f346824d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:44:07.026 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:44:06 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-f[62124]: 2024-04-21T10:44:06.716+0000 7f346824d700 -1 mds.f *** got signal Terminated *** 2024-04-21T10:44:09.807 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:44:09 smithi190 podman[69555]: 2024-04-21 10:44:09.533153703 +0000 UTC m=+2.921493430 container died 2a1ab1cd00df3d1f71b56395747e4908593e8873db2d203e68bcdbf926b01ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-f, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T10:44:10.174 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:44:09 smithi190 podman[69555]: 2024-04-21 10:44:09.891334025 +0000 UTC m=+3.279673760 container cleanup 2a1ab1cd00df3d1f71b56395747e4908593e8873db2d203e68bcdbf926b01ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-f, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-21T10:44:10.174 INFO:journalctl@ceph.mds.f.smithi190.stdout:Apr 21 10:44:09 smithi190 bash[69555]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-f 2024-04-21T10:44:10.311 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.f.service' 2024-04-21T10:44:10.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:44:10.343 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-21T10:44:10.343 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-21T10:44:10.343 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.i 2024-04-21T10:44:10.526 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:44:10 smithi190 systemd[1]: Stopping Ceph mds.i for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:44:11.026 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:44:10 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i[62749]: 2024-04-21T10:44:10.661+0000 7f4a94500700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:44:11.026 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:44:10 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i[62749]: 2024-04-21T10:44:10.661+0000 7f4a94500700 -1 mds.i *** got signal Terminated *** 2024-04-21T10:44:13.861 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:44:13 smithi190 podman[69676]: 2024-04-21 10:44:13.535425526 +0000 UTC m=+2.975775988 container died fc172937a855998c9b00e3851472e828b6e8a35fa7314ecb3e9ec9768f637bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-21T10:44:14.152 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:44:13 smithi190 podman[69676]: 2024-04-21 10:44:13.961343458 +0000 UTC m=+3.401693927 container cleanup fc172937a855998c9b00e3851472e828b6e8a35fa7314ecb3e9ec9768f637bd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-21T10:44:14.153 INFO:journalctl@ceph.mds.i.smithi190.stdout:Apr 21 10:44:13 smithi190 bash[69676]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-i 2024-04-21T10:44:14.363 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.i.service' 2024-04-21T10:44:14.394 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:44:14.394 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-21T10:44:14.395 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-21T10:44:14.395 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.l 2024-04-21T10:44:14.713 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:44:14 smithi190 systemd[1]: Stopping Ceph mds.l for b77245c8-ffc8-11ee-bc93-c7b262605968... 2024-04-21T10:44:15.029 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:44:14 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-l[61812]: 2024-04-21T10:44:14.714+0000 7fcee24f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T10:44:15.029 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:44:14 smithi190 ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-l[61812]: 2024-04-21T10:44:14.714+0000 7fcee24f5700 -1 mds.l *** got signal Terminated *** 2024-04-21T10:44:18.414 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:44:18 smithi190 podman[69799]: 2024-04-21 10:44:18.077031845 +0000 UTC m=+3.465430632 container died 68236a017c6d51094348f5660c6f5dec09140f8b839d845ef07c0b889965ce52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T10:44:18.707 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:44:18 smithi190 podman[69799]: 2024-04-21 10:44:18.507407984 +0000 UTC m=+3.895806770 container cleanup 68236a017c6d51094348f5660c6f5dec09140f8b839d845ef07c0b889965ce52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-21T10:44:18.707 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:44:18 smithi190 bash[69799]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-l 2024-04-21T10:44:18.938 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.l.service' 2024-04-21T10:44:18.957 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:44:18 smithi190 podman[69813]: 2024-04-21 10:44:18.708607976 +0000 UTC m=+0.628399422 container remove 68236a017c6d51094348f5660c6f5dec09140f8b839d845ef07c0b889965ce52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-b77245c8-ffc8-11ee-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T10:44:18.958 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:44:18 smithi190 systemd[1]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.l.service: Deactivated successfully. 2024-04-21T10:44:18.958 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:44:18 smithi190 systemd[1]: Stopped Ceph mds.l for b77245c8-ffc8-11ee-bc93-c7b262605968. 2024-04-21T10:44:18.958 INFO:journalctl@ceph.mds.l.smithi190.stdout:Apr 21 10:44:18 smithi190 systemd[1]: ceph-b77245c8-ffc8-11ee-bc93-c7b262605968@mds.l.service: Consumed 1.096s CPU time. 2024-04-21T10:44:18.967 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T10:44:18.968 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-21T10:44:18.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T10:44:19.167 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:44:21.731 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T10:44:21.920 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:44:24.351 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T10:44:24.540 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:44:26.813 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T10:44:26.846 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T10:44:26.877 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T10:44:26.903 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-21T10:44:26.905 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584/remote/smithi026/crash 2024-04-21T10:44:26.906 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/crash -- . 2024-04-21T10:44:26.941 INFO:teuthology.orchestra.run.smithi026.stderr:tar: /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T10:44:26.942 INFO:teuthology.orchestra.run.smithi026.stderr:tar: Error is not recoverable: exiting now 2024-04-21T10:44:26.943 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584/remote/smithi169/crash 2024-04-21T10:44:26.944 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/crash -- . 2024-04-21T10:44:26.981 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T10:44:26.981 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now 2024-04-21T10:44:26.983 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584/remote/smithi190/crash 2024-04-21T10:44:26.984 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/crash -- . 2024-04-21T10:44:27.016 INFO:teuthology.orchestra.run.smithi190.stderr:tar: /var/lib/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T10:44:27.016 INFO:teuthology.orchestra.run.smithi190.stderr:tar: Error is not recoverable: exiting now 2024-04-21T10:44:27.017 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-21T10:44:27.017 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-21T10:44:27.054 INFO:tasks.cephadm:Compressing logs... 2024-04-21T10:44:27.055 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T10:44:27.098 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T10:44:27.101 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T10:44:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.log 2024-04-21T10:44:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:: No such file or directory 2024-04-21T10:44:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T10:44:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.g.log 2024-04-21T10:44:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.log: /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.j.log 2024-04-21T10:44:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.3.log 2024-04-21T10:44:27.142 INFO:teuthology.orchestra.run.smithi190.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-21T10:44:27.142 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.log 2024-04-21T10:44:27.143 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.2.log 2024-04-21T10:44:27.143 INFO:teuthology.orchestra.run.smithi190.stderr:gzip/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.f.log 2024-04-21T10:44:27.143 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T10:44:27.143 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.5.log 2024-04-21T10:44:27.144 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.c.log 2024-04-21T10:44:27.144 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mon.c.log 2024-04-21T10:44:27.144 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mgr.z.log 2024-04-21T10:44:27.144 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.i.log 2024-04-21T10:44:27.145 INFO:teuthology.orchestra.run.smithi190.stderr: 87.3% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.log.gz/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mgr.z.log: 2024-04-21T10:44:27.145 INFO:teuthology.orchestra.run.smithi190.stderr: 90.5% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T10:44:27.145 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T10:44:27.145 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.11.log 2024-04-21T10:44:27.146 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T10:44:27.146 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.8.log 2024-04-21T10:44:27.146 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.11.log: 91.9% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-21T10:44:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.0.log 2024-04-21T10:44:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mgr.x.log 2024-04-21T10:44:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mon.a.log 2024-04-21T10:44:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.6.log 2024-04-21T10:44:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mon.a.log: 87.3% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T10:44:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T10:44:27.148 INFO:teuthology.orchestra.run.smithi026.stderr: 90.3% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T10:44:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.6.log: 94.7% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-21T10:44:27.148 INFO:teuthology.orchestra.run.smithi026.stderr: 94.6% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-21T10:44:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.a.log 2024-04-21T10:44:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T10:44:27.149 INFO:teuthology.orchestra.run.smithi026.stderr: 84.1% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T10:44:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.d.log 2024-04-21T10:44:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.9.log 2024-04-21T10:44:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-21T10:44:27.149 INFO:teuthology.orchestra.run.smithi026.stderr: 94.5% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-21T10:44:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T10:44:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T10:44:27.150 INFO:teuthology.orchestra.run.smithi026.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T10:44:27.151 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T10:44:27.152 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-21T10:44:27.152 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.log 2024-04-21T10:44:27.152 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T10:44:27.152 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.7.log 2024-04-21T10:44:27.152 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.b.log 2024-04-21T10:44:27.152 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.4.log 2024-04-21T10:44:27.152 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.1.log 2024-04-21T10:44:27.153 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mgr.y.log 2024-04-21T10:44:27.153 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.h.log 2024-04-21T10:44:27.153 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose 87.3% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T10:44:27.153 INFO:teuthology.orchestra.run.smithi169.stderr: -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.10.log 2024-04-21T10:44:27.153 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.cephadm.log 90.5% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T10:44:27.153 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-04-21T10:44:27.153 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.10.log: /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.e.log 2024-04-21T10:44:27.153 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T10:44:27.153 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T10:44:27.154 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.e.log: 91.9% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-21T10:44:27.154 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mon.b.log 2024-04-21T10:44:27.154 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.k.log 2024-04-21T10:44:27.154 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T10:44:27.156 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.l.log 2024-04-21T10:44:27.163 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T10:44:27.167 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T10:44:27.172 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-21T10:44:27.176 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-21T10:44:27.181 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T10:44:27.183 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T10:44:27.184 INFO:teuthology.orchestra.run.smithi190.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T10:44:27.202 INFO:teuthology.orchestra.run.smithi169.stderr: 94.3% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-21T10:44:27.278 INFO:teuthology.orchestra.run.smithi169.stderr: 93.9% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-21T10:44:27.374 INFO:teuthology.orchestra.run.smithi026.stderr: 90.3% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-21T10:44:27.429 INFO:teuthology.orchestra.run.smithi169.stderr: 92.2% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-21T10:44:27.545 INFO:teuthology.orchestra.run.smithi190.stderr: 92.1% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-21T10:44:27.584 INFO:teuthology.orchestra.run.smithi190.stderr: 94.8% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-21T10:44:27.994 INFO:teuthology.orchestra.run.smithi026.stderr: 90.5% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-21T10:44:28.070 INFO:teuthology.orchestra.run.smithi169.stderr: 93.4% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-21T10:44:28.304 INFO:teuthology.orchestra.run.smithi026.stderr: 93.2% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-21T10:44:28.314 INFO:teuthology.orchestra.run.smithi169.stderr: 93.3% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-21T10:44:28.315 INFO:teuthology.orchestra.run.smithi169.stderr: 95.5% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-21T10:44:28.358 INFO:teuthology.orchestra.run.smithi169.stderr: 93.4% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-21T10:44:28.402 INFO:teuthology.orchestra.run.smithi026.stderr: 93.3% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-21T10:44:28.444 INFO:teuthology.orchestra.run.smithi026.stderr: 93.4% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-21T10:44:28.548 INFO:teuthology.orchestra.run.smithi169.stderr: 93.5% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-21T10:44:28.550 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-04-21T10:44:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m1.436s 2024-04-21T10:44:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:user 0m6.274s 2024-04-21T10:44:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m0.246s 2024-04-21T10:44:28.551 INFO:teuthology.orchestra.run.smithi190.stderr: 93.3% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-21T10:44:28.568 INFO:teuthology.orchestra.run.smithi190.stderr: 93.3% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-21T10:44:28.608 INFO:teuthology.orchestra.run.smithi190.stderr: 93.6% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-21T10:44:28.615 INFO:teuthology.orchestra.run.smithi026.stderr: 93.4% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-21T10:44:28.617 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-21T10:44:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:real 0m1.505s 2024-04-21T10:44:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:user 0m6.213s 2024-04-21T10:44:28.618 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m0.223s 2024-04-21T10:44:28.651 INFO:teuthology.orchestra.run.smithi190.stderr: 93.3% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-21T10:44:31.108 INFO:teuthology.orchestra.run.smithi190.stderr: 93.9% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-21T10:44:38.333 INFO:teuthology.orchestra.run.smithi190.stderr: 93.3% -- replaced with /var/log/ceph/b77245c8-ffc8-11ee-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-21T10:44:38.334 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-04-21T10:44:38.335 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m11.219s 2024-04-21T10:44:38.335 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m21.209s 2024-04-21T10:44:38.335 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m0.710s 2024-04-21T10:44:38.335 INFO:tasks.cephadm:Archiving logs... 2024-04-21T10:44:38.336 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584/remote/smithi026/log 2024-04-21T10:44:38.337 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T10:44:39.214 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584/remote/smithi169/log 2024-04-21T10:44:39.215 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T10:44:40.042 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584/remote/smithi190/log 2024-04-21T10:44:40.043 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T10:44:42.902 INFO:tasks.cephadm:Removing cluster... 2024-04-21T10:44:42.903 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 --force 2024-04-21T10:44:43.102 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:44:43.370 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 --force 2024-04-21T10:44:43.562 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:44:43.813 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b77245c8-ffc8-11ee-bc93-c7b262605968 --force 2024-04-21T10:44:44.003 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: b77245c8-ffc8-11ee-bc93-c7b262605968 2024-04-21T10:44:44.273 INFO:tasks.cephadm:Removing cephadm ... 2024-04-21T10:44:44.273 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T10:44:44.291 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T10:44:44.306 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T10:44:44.321 INFO:tasks.cephadm:Teardown complete 2024-04-21T10:44:44.321 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-21T10:44:44.332 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-21T10:44:44.332 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T10:44:44.334 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T10:44:44.349 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T10:44:44.456 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-21T10:44:44.465 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-21T10:44:44.465 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T10:44:44.468 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T10:44:44.470 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T10:44:44.481 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-04-21T10:44:44.482 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-21T10:44:44.484 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-04-21T10:44:44.487 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T10:44:44.487 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-04-21T10:44:44.487 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 +101us[ +94us] +/- 81ms 2024-04-21T10:44:44.488 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -623us[ -630us] +/- 61ms 2024-04-21T10:44:44.488 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 +783us[ +783us] +/- 88ms 2024-04-21T10:44:44.488 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:44:44.488 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T10:44:44.488 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-21T10:44:44.488 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 +94us[ +105us] +/- 81ms 2024-04-21T10:44:44.488 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 -625us[ -614us] +/- 61ms 2024-04-21T10:44:44.488 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +776us[ +786us] +/- 88ms 2024-04-21T10:44:44.489 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:44:44.489 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T10:44:44.490 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-04-21T10:44:44.490 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 32 +91us[ +91us] +/- 81ms 2024-04-21T10:44:44.490 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -636us[ -632us] +/- 61ms 2024-04-21T10:44:44.490 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +787us[ +787us] +/- 88ms 2024-04-21T10:44:44.490 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T10:44:44.490 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-21T10:44:44.500 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-21T10:44:44.501 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-21T10:44:44.510 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T10:44:44.603 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T10:44:44.667 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T10:44:44.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T10:44:44.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:44:44.759 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T10:44:44.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:44:44.785 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T10:44:44.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:44:44.812 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-21T10:44:44.823 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-21T10:44:44.833 INFO:teuthology.task.internal:Duration was 1834.601750 seconds 2024-04-21T10:44:44.833 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-21T10:44:44.843 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-21T10:44:44.844 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T10:44:44.846 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T10:44:44.849 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T10:44:44.883 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T10:44:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T10:44:44.887 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T10:44:45.233 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-21T10:44:45.233 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-04-21T10:44:45.233 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T10:44:45.284 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-04-21T10:44:45.284 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T10:44:45.310 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-04-21T10:44:45.311 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T10:44:45.331 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-21T10:44:45.331 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T10:44:45.333 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T10:44:45.353 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T10:44:45.395 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-21T10:44:45.396 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T10:44:45.400 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T10:44:45.418 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T10:44:45.778 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-21T10:44:45.790 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-21T10:44:45.791 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T10:44:45.815 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T10:44:45.838 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T10:44:45.862 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-21T10:44:45.872 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T10:44:45.875 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T10:44:45.881 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T10:44:45.895 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-04-21T10:44:45.900 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-04-21T10:44:45.923 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-04-21T10:44:45.950 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T10:44:45.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:44:45.964 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T10:44:45.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:44:45.978 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T10:44:46.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T10:44:46.003 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-21T10:44:46.015 INFO:teuthology.task.internal:Transferring archived files... 2024-04-21T10:44:46.015 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584/remote/smithi026 2024-04-21T10:44:46.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T10:44:46.055 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584/remote/smithi169 2024-04-21T10:44:46.055 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T10:44:46.089 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584/remote/smithi190 2024-04-21T10:44:46.089 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T10:44:46.123 INFO:teuthology.task.internal:Removing archive directory... 2024-04-21T10:44:46.124 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T10:44:46.126 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T10:44:46.128 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T10:44:46.172 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-21T10:44:46.186 INFO:teuthology.task.internal:Not uploading archives. 2024-04-21T10:44:46.187 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-21T10:44:46.198 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-21T10:44:46.199 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T10:44:46.201 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T10:44:46.203 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T10:44:46.213 INFO:teuthology.orchestra.run.smithi026.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 10:44 /home/ubuntu/cephtest 2024-04-21T10:44:46.215 INFO:teuthology.orchestra.run.smithi169.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 10:44 /home/ubuntu/cephtest 2024-04-21T10:44:46.227 INFO:teuthology.orchestra.run.smithi190.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 10:44 /home/ubuntu/cephtest 2024-04-21T10:44:46.228 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-21T10:44:46.240 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-21T10:44:46.266 INFO:teuthology.nuke:Checking targets against current locks 2024-04-21T10:44:46.291 DEBUG:teuthology.nuke:shortname: smithi026 2024-04-21T10:44:46.291 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T10:44:46.315 DEBUG:teuthology.nuke:shortname: smithi169 2024-04-21T10:44:46.315 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T10:44:46.320 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 10:08:12.875657', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ2lqbpudOBSVXMl416tiMvWri1zUhLjXS6Y/XKmRBkP4X8ME/S+lHPQhzNdM+J4GV9EfZmRdB7g8NuxP9xzn00='} 2024-04-21T10:44:46.338 DEBUG:teuthology.nuke:shortname: smithi190 2024-04-21T10:44:46.338 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T10:44:46.342 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 10:08:12.880022', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP2S0u3x+CMOOaG2F5tb2vdbByndlfINEKpgkpH3o5XiNOc7UKoiytaEAS6e1c3qyt1H9wMGN4hL0DyTHKMhlJ0='} 2024-04-21T10:44:46.360 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_09:54:56-fs-wip-lusov-quiescer-distro-default-smithi/7666584', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 10:08:12.877501', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCCXjrNp0exN6HvNy3i+osWJ6TFiX6gN8XIIvSeLyo53AGUJdyJBLQI+csy8oiYwWOzcFQswOtx+Y8MocvONLug='} 2024-04-21T10:44:46.370 INFO:teuthology.orchestra.console.smithi026:Power off 2024-04-21T10:44:46.370 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T10:44:46.396 DEBUG:teuthology.orchestra.console.smithi026:power off output: Chassis Power Control: Down/Off 2024-04-21T10:44:46.396 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T10:44:46.412 INFO:teuthology.orchestra.console.smithi169:Power off 2024-04-21T10:44:46.412 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T10:44:46.425 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-04-21T10:44:46.434 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off 2024-04-21T10:44:46.435 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T10:44:46.451 INFO:teuthology.orchestra.console.smithi190:Power off 2024-04-21T10:44:46.451 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T10:44:46.462 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-21T10:44:46.475 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2024-04-21T10:44:46.475 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T10:44:46.498 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-04-21T10:44:50.426 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T10:44:50.462 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T10:44:50.500 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T10:44:50.537 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-04-21T10:44:50.573 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-21T10:44:50.610 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-04-21T10:44:54.538 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T10:44:54.574 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T10:44:54.610 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T10:44:54.650 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is off 2024-04-21T10:44:54.685 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2024-04-21T10:44:54.719 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-04-21T10:44:54.751 INFO:teuthology.orchestra.console.smithi026:Power off completed 2024-04-21T10:44:54.786 INFO:teuthology.orchestra.console.smithi169:Power off completed 2024-04-21T10:44:58.721 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T10:44:58.832 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2024-04-21T10:44:58.932 INFO:teuthology.orchestra.console.smithi190:Power off completed 2024-04-21T10:44:59.039 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 1834.6017496585846 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi026 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b8ddeda347de6c45510a6df67440a1d298c37030 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=127650b1201b47b19c9fd22f7fd5526f status: fail success: false 2024-04-21T10:44:59.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-21T10:44:59.172 INFO:teuthology.run:FAIL